Skip to content

A simple downloader/processor for CrunchBase API data, built amongst two pals

Notifications You must be signed in to change notification settings

ephekt/PeopleFinder

Folders and files

NameName
Last commit message
Last commit date

Latest commit

 

History

18 Commits
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 

Repository files navigation

Download, Crunch, and Comb your CrunchBase data!

Create a file called api.key with your API Key.

Then: bundle install ruby downloader.rb ruby cruncher.rb ruby comb.rb ... or something like that.

File Structure:

Path: /data Notes: Ignored in the .gitignore. Holds information from fetching JSON files from Crunchbase.com/Api

Path: /resources Notes: Just some documents to save related to analysis and other things on top of the core code base

Files:

common.rb - some shared monkey patches and constants used by other files downloader.rb - parallel fetching of crunchbase people and companies in json format, stores to /data cruncher.rb - loads crunchbase data into mongodb. will read from /data, data will not be overwritten into MongoDB. exploration/comb.rb - sift through people and pull out data

Future notes:

It'd be good to timestamp the data directories so we can do pulls daily and then run the cruncher on the new directories, by date.

About

A simple downloader/processor for CrunchBase API data, built amongst two pals

Resources

Stars

Watchers

Forks

Releases

No releases published

Packages

No packages published

Languages