Shell script for Google search result parsing

This is the shell script I wrote to help me perform the analysis I did for Quest 5.

1. Perform a site:yoursite.edu search in Google, displaying 100 results per page.
2. Save each page (Google will only give you 10 at most) into a folder named yoursite.edu
3. Download the shell script to the directory that contains the yoursite.edu directory.
4. At the command prompt, type:

./google-results-parse yoursite.edu

5. OR, if you named the yoursite.edu directory something different, run this:

./google-results-parse yoursite.edu savedresultsdirectory

6. It will create a "savedresultsdirectory-parsed" directory, which will contain a "domainlist" file and a "pagelinks" directory. The "domainlist" gives the subdomain breakdown of the search results.  The "pagelinks" folder contains files for each subdomain that include all of the search result URLs for that subdomain.

Download the file here.

Open Ed. Quest 5 -- Searching for a Better Way (to Search)

Quest 5

"Many BYU faculty already openly share their syllabi and other course materials on personal websites, through iTunesU, and through other mechanisms ... Find as many of the open educational resources being shared by BYU faculty as you can..."

It seems to me that discoverability is really going to be the ultimate make-or-break hinge issue for OER.  One could produce world class, high quality OER that trumps everything that any institutional OER effort produces, and yet remain in complete obscurity with no hope of ever actually sharing these wonderful OER with anyone at all.  And after all, if you take the time and trouble to make some kind of resource with openness in mind, it seems silly to have it be completely worthless (or at least, gravely underused) in the end because you weren't able to put it somewhere that people would find it.

This post isn't going to discuss the hows and whys of publishing open educational content for maximum discoverability. We'll save that for another time.  However, Quest 5 gives us the specific assignment to comb over BYU's web presence looking for faculty-produced OER content, and it begs the question, "How would one go about finding all of the OER on a university's web space?"

The task is not trivial.

Quest 2 - For Real Now

Up to this point in the course we've done a lot of talking.  We've had great discussions about the history of the open education movement, usage rights, sustainability models, reusability, remixability--even hippies!--and just about everything in between.  But now it's time to get to work!  No more talking!  This is going to be fun.

So, looking ahead to Quest 6, we in our guilds will need to collaborate to create a course entirely out of open educational resources.  Because we have so little time left, we decided as a class that we would devote everything we do in the remaining quests to work toward our goals for Quest 6.

The course that we as a class originally (more on that later) decided to build is 10th grade social studies--World Civilizations.  We will attempt to build this course entirely from OERs that meet the Utah State K-12 Core Curriclum Standards for World Civilizations.

So I decided that for Quest 2 I would just jump into the pool and do my best to find as many OERs as possible that could help us meet the objectives and standards set out in the Utah K-12 Core.  This actually turned out to be the first time I've ever made a real attempt to collect a large number of OERs from multiple repositories for a single purpose.  Everything that follows here is a description of my first purposeful experience looking for OERs.

If at any point you feel like cutting to the chase, click on this link or just scroll to the end.

History and Status of OER at BYU (DRAFT)

While the social climate for a policy of openness at BYU is currently in its nascent and humble stages, it is quickly gaining momentum.  Despite the lack of an official university-wide “open” policy at BYU, a number of open activities are already well under way.  To date, a growing number of OERs and OER-like resources are already offered through various channels on campus.

BYU Independent Study is one example of an organizational effort to contribute to the OER community.  IS already offers free access to a selection of their courses for personal development.  Additionally, IS will soon launch a pilot program to offer several of their most popular courses to all BYU students and the general open education community in a manner similar to OER initiatives currently underway at other universities.

One more such organizational effort is based in the McKay School of Education, which recently launched a pilot program to openly publish (with voluntary instructor participation) several School of Education courses.

In addition to these organizational efforts, many BYU faculty have, as individuals, posted personally developed course content in various ways to make these resources available to the general public.  Some faculty post course syllabi, reading lists, and other course content on personal websites, while other faculty have even recorded their lectures and published them on well known online media repositories such as iTunes and YouTube.

While each of these efforts is a step in the direction of openness, the efforts are typically isolated from each other.  The real power behind these efforts will be found in strength of numbers and unity of purpose.  Because many departments and individuals at BYU are already beginning to contribute to the OER movement, there is a clear need for a central repository of BYU-produced OERs.  The Center for Teaching and Learning is clearly the organization with the resources and momentum toachieve such an undertaking.