Skip to content

Commit

Permalink
Created interface with Meetup.com API to get upcoming events. Updated…
Browse files Browse the repository at this point in the history
… documentation.
  • Loading branch information
JangoSteve committed Aug 27, 2014
1 parent 41e1f95 commit cea066a
Show file tree
Hide file tree
Showing 4 changed files with 108 additions and 164 deletions.
1 change: 1 addition & 0 deletions .ruby-version
Original file line number Diff line number Diff line change
@@ -0,0 +1 @@
2.1.2
80 changes: 44 additions & 36 deletions README.md
Original file line number Diff line number Diff line change
@@ -1,19 +1,22 @@
# Alfa Jango Count Proxy
# A2RB Count Proxy

This application grabs download and install counts for Alfa Jango's
open-source projects from various site APIs, as a proxy for
http://os.alfajango.com.
Forked and built from the [Alfa Jango Count
Proxy](https://github.com/alfajango/count-proxy).

This application grabs Meetup info from the Meetup API for the Ann Arbor
Ruby Brigade group, and stores it in a static JSONP file on Amazon S3 to
be embedded directly in a2rb.org.

## Process

This script runs a few times a day to update the project counts.
This script runs a few times a day to update the Meetup group info.

It will loop through each project, with a slight delay to ease load on
the vendor servers. For each project, it will grab the download/install
counts from each vendor's API.
It will loop through each event, with a slight delay to ease load on
the vendor servers. For each event, it will grab the event name,
description, date, location, and number of people registered.

The script will then write the counts for all projects and
vendors to a JavaScript JSONP string and store it as a file on Amazon S3.
The script will then write the info for all events to a JavaScript JSONP
string and store it as a file on Amazon S3.

Writing to jsonp will eliminate cross-domain concerns inherent in
loading JSON data from S3.
Expand All @@ -24,44 +27,49 @@ Example file format for the json file stored on Amazon S3:

```js
setJSON({
easytabs: {
github: {
watchers: 100,
forks: 20
},
jspkg: {
total_downloads: 6000
}
},
jquery-rails: {
github: {
watchers: 300,
forks: 100
},
rubygems: {
total_downloads: 3000000
}
Ann-Arbor-Ruby: {
events: [
{
name: "Event Name",
description: "<p>Event description</p>",
time: 1412118000000,
status: "upcoming",
venue: {
name: "Venue Name",
address_1: "Address 1",
address_2: "Address 2",
city: "Ann Arbor",
state: "MI",
zip: "48105"
}
yes_rsvp_count: 56,
maybe_rsvp_count: 2,
headcount: 0,
event_url: "http://www.meetup.com/Ann-Arbor-Ruby/events/203156012/",
...
}
]
}
});
```

## Running script

First, update the top of the script with your own settings for the projects and
services you want to import, as well as with your own S3 bucket name and
desired filenames.
First, update the top of the script with the settings for the Meetup
group and and Meetup API credentials to import, as well as with your own
S3 bucket name and desired filenames.

To run the script, make sure it's executable:

```
sudo chmod u+x bin/get_counts
```

Then run it, be sure that your `AWS_ACCESS_KEY` and `AWS_SECRET_KEY`
Then run it, be sure that your `MEETUP_API_KEY`, `AWS_ACCESS_KEY`, and `AWS_SECRET_KEY`
variables are present in your environment:

```
AWS_ACCESS_KEY_ID=YOUR_ACCESS_KEY_HERE AWS_SECRET_ACCESS_KEY=YOUR_SECRET_KEY_HERE bin/get_counts
MEETUP_API_KEY=YOUR_MEETUP_API_KEY_HERE AWS_ACCESS_KEY_ID=YOUR_ACCESS_KEY_HERE AWS_SECRET_ACCESS_KEY=YOUR_SECRET_KEY_HERE bin/get_events
```

## Setting up on Heroku
Expand All @@ -76,21 +84,21 @@ Just clone this repo from Github, and create a new app on Heroku's Cedar
stack with the heroku gem:

```
heroku apps:create my-count-proxy
heroku apps:create my-meetup-proxy
git push heroku
```

Then add your Amazon Web Services access key and secret key to Heroku's
environment config for the app:

```
heroku config:add AWS_ACCESS_KEY_ID=YOUR_ACCESS_KEY_HERE AWS_SECRET_ACCESS_KEY=YOUR_SECRET_KEY_HERE
heroku config:add MEETUP_API_KEY=YOUR_MEETUP_API_KEY_HERE AWS_ACCESS_KEY_ID=YOUR_ACCESS_KEY_HERE AWS_SECRET_ACCESS_KEY=YOUR_SECRET_KEY_HERE
```

To test the script on heroku:

```
heroku run get_counts
heroku run get_events
```

Now, let's setup Heroku's "scheduler" addon (previously called the "cron" addon):
Expand All @@ -106,6 +114,6 @@ heroku addons:open scheduler
```

The above will open the Heroku scheduler dashboard in your browser,
where you can click "Add job...", type in `get_counts`, and select your
where you can click "Add job...", type in `get_events`, and select your
frequency (probably daily, or at most hourly, if possible, to minimize
impact on venders' available API resources).
128 changes: 0 additions & 128 deletions bin/get_counts

This file was deleted.

63 changes: 63 additions & 0 deletions bin/get_events
Original file line number Diff line number Diff line change
@@ -0,0 +1,63 @@
#!/usr/bin/env ruby

# Use Bundler to use correct gems/dependencies
require "rubygems"
require "bundler/setup"

require 'open-uri'
require 'json'
require 'openssl'
require 'aws/s3'

# Hack to skip ssl certificate validation error with github api
OpenSSL::SSL::VERIFY_PEER = OpenSSL::SSL::VERIFY_NONE

FILENAME = "output.json"
S3_BUCKET = 's3.alfajango.com'
S3_FILENAME = 'a2rb-meetup-events.json' # or simply, FILENAME
SLEEP = 1.0

GROUPS = ["Ann-Arbor-Ruby"]

puts "Starting import..."

output = {}.tap do |out|
GROUPS.each do |group|
print " #{group}:"
out[group.to_s] = {"events" => []}
uri = "https://api.meetup.com/2/events?&sign=true&photo-host=public&group_urlname=#{group}&page=20&key=#{ENV['MEETUP_API_KEY']}"

begin
json = URI.parse(uri).open.read
parsed_json = JSON.parse(json)

out[group.to_s]["events"] = parsed_json["results"]
rescue Exception => e
puts "\nEncountered exception, skipping #{group}[events]: #{e}"
next
end

puts ""
sleep SLEEP
end
end

puts "Writing file #{FILENAME}..."
File.open(FILENAME, 'w') { |f|
f.write('setJSON(' + output.to_json + ');')
}

puts "Uploading to #{S3_FILENAME} in S3 bucket #{S3_BUCKET}..."
AWS::S3::Base.establish_connection!(
:access_key_id => ENV['AWS_ACCESS_KEY_ID'],
:secret_access_key => ENV['AWS_SECRET_ACCESS_KEY']
)
AWS::S3::S3Object.store(
S3_FILENAME,
open(FILENAME),
S3_BUCKET,
content_type: 'application/javascript',
access: :public_read
)

puts "Done!"

0 comments on commit cea066a

Please sign in to comment.