Home
The college is receiving an increasing number of requests to provide college data to external vendors who have been contracted to provide services for our students. Responding to these requests manually can adversely impact other IT tasks as employees are interrupted from their primary tasks.
- Respond (i.e. provide the data) quickly to approved data export requests.
- Reduce time spent by skilled IT workers responding to data export requests.
- Provide a standardized, documented way that the college provides data to external vendors.
The Data Export service should be flexible enough to accommodate a variety of data export requirements (both the data itself and how it is delivered).
- Automatically generate and deliver, or stage, requested data in the format required.
- Ideally, responding to new requests should only require:
- Creating a SQL view that retrieves the data.
- Adding configuration settings that define the export.
- (No changes to the code.)
The initial version of the Data Export service was developed against the requirements for a specific 3rd-party request, within the context of the overall requirements outlined above.
- ASP.NET MVC3 (.NET 4) - as a Web Service
- Crontab - Schedules calls to the web service.
- SFTP (SSH) - For uploading to the vendor.
- via the SSH.NET library.
While the initial need for this application is to provide a data to a specific 3rd-party, the architecture is designed to generically support a variety of data export needs. Ideally, adding a new export will simply be a matter of adding the appropriate configuration settings and should not require any additional coding.
Once you've configured your data export(s) (see Configuration), you can invoke them by (including formatting and delivery of the data) simply by visiting the URL associated with the desired export. For example, to perform an export for the configured exporter named MyExport, visit the following URL:
https://YOUR_SERVER_DOMAIN/dataexport/Api/Export/MyExport
At Bellevue College, we use crontab and wget to schedule these HTTP calls.
Although the application is intended to run without human input, a simple interface can be found at https://YOUR_SERVER_DOMAIN/dataexport/. This interface is provided primarily for testing purposes.
NOTE:
These processes often take a long time to execute, so it may be some time before a valid response is returned to the browser.
The links listed were manually added via static HTML. Future plans include dynamically generating HTML links on this interface from the configured exporters. (See Configuration.) For the time-being, you may simply modify the links provided on this page to correspond to your the exporters you have configured.
Logging of internal application activity is handled by the ELMAH library. To view the application log, visit
https://YOUR_SERVER_DOMAIN/dataexport/elmah.axd