Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

DataLayer to provide data from LIBSVM format file #1571

Closed
wants to merge 11 commits into from

Conversation

tnarihi
Copy link
Contributor

@tnarihi tnarihi commented Dec 13, 2014

I've just implemented a new layer which provides data from LIBSVM format file.

<label> <index1>:<feature1> ... <index M>:<feature M>

Here is an example of using this layer: http://nbviewer.ipython.org/github/tnarihi/caffe/blob/libsvmdatalayer/examples/libsvm_format_data_into_caffe.ipynb

@sguada
Copy link
Contributor

sguada commented Dec 13, 2014

Adding more data layers is not a sustainable approach. Please take a look at #1568 and try to simplify your approach.

@shelhamer
Copy link
Member

@tnarihi thanks for adding another data format, but as @sguada mentioned we are trying to come up with a more general, less redundant data pipeline. At the same time we are working on making it easier to have "personal" layers that will build with Caffe but don't need to be merged in the main project.

Your libsvm layer could be one of these contributed layers.

@tnarihi
Copy link
Contributor Author

tnarihi commented Dec 17, 2014

@sguada @shelhamer Thanks for the comments ! More general data layer sounds good. I will take a look at the PRs. The "personal" layer concept is also interesting. Is there any PR regarding it?

@shelhamer
Copy link
Member

@tnarihi this could be done by a Python data layer #1703 or through de-centralized / personal layers as discussed in #1896 with a proof-of-concept in #2036. Closing to avoid adding more data layers since the data pipeline needs further work and special formats can be handled by Python. Thanks for sharing your code!

@shelhamer shelhamer closed this Mar 10, 2015
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
None yet
Projects
None yet
Development

Successfully merging this pull request may close these issues.

3 participants