OpenMindat: A Python Package for Geomaterial Data Analysis and Retrieval from Mindat API

The OpenMindat Python package is designed to facilitate querying and retrieving data on minerals and geomaterials from the Mindat API. It provides classes for detailed queries based on various attributes like IMA status, keywords, and specific geomaterial properties.

GitHub Repository: OpenMindat Python Package

Table of Contents

• Get Started
• Endpoint Descriptions
• Use Cases
  • 0. Setup
  • 1. Perform Detailed Queries on Geomaterials
  • 2. Retrieve IMA-Approved Minerals
  • 3. Search Geomaterials Using Keywords
  • 4. Retrieve Localities
  • 5. Retrieve Type Localities for IMA-Approved Mineral Species
  • 6. Retrieve Locality Occurrences for Single Mineral Species
• Documentation and Relevant Links
• Contact Us
• License
• Authors
• Citations
• Acknowledgments
• Upgrade Logs

Get Started

Install via Pip

foo@bar:~$ pip install openmindat

Import the Package in Python

import openmindat

Endpoint Descriptions

Endpoint	Classes	Description
Dana8	- DanaRetriever()	Search query to return information about the Dana-8 classification standard.
Geomaterials	- GeomaterialRetriever() - GeomaterialIdRetreiver() - GeomaterialDictRetriever()	Search query to return information about mindat database items such as id, name, group id etc.
Geomaterial_search	- GeomaterialSearchRetriever()	Query to search for mindat database entries based on search keywords.
Localities	- LocalitiesRetriever() - LocalitiesIdRetriever()	Search query to return information about different localities, for examples the main elements present in the Jegdalek ruby deposit in Afghanistan
Localities_Age	- LocalitiesAgeRetriever() - LocalitiesAgeIdRetriever()	Search query to return locality age details.
Localities_Statues	- LocalitiesStatusRetriever() - LocalitiesStatusIdRetriever()	Search query to return information about the status type of localities. For example, abandoned is a status type.
Localities_Type	- LocalitiesTypeRetriever() - LocalitiesTypeIdRetriever()	Search query to return information about the type of localities. For example, a Mining Field is a locality type.
LocGeoregion2	- GeoRegionRetriever()	Gives information about GeoRegion boundaries
LocObject	- LocobjectRetriever()	N/A
Minerals-IMA	- MineralsIMARetriever() - MineralsIdRetriever()	Search query to return IMA details for a mineral. For example the year it's IMA status was approved.
Nickel_Strunz	- StrunzRetriever()	Search query to return information about the Nickel-Strunz-10 classification standard.
Photo_Count	- PhotoCountRetriever()	N/A

Use Cases

0. Setup

Setting API Key in Alternative Ways

import os

os.environ["MINDAT_API_KEY"] = 'Your_Mindat_API_Key'

If you do not have a Mindat API key, please refer to How to Get My Mindat API Key or Token?

You can also set the API key by following the general queries.

Checking Available Methods

from openmindat import GeomaterialRetriever

gr = GeomaterialRetriever()
# Print out the available functions for a class
gr.available_methods()

from openmindat import GeomaterialRetriever

gr = GeomaterialRetriever()
# Typo check
gr.elements_in('Cu')
'''>>> AttributeError: 'GeomaterialRetriever' object has no attribute 'elements_in', 
Available methods: ['_init_params', 'available_methods', 'bi_max', 'bi_min', 'cleavagetype', 'color', 
'colour', 'crystal_system', 'density_max', 'density_min', 'diaphaneity', 'elements_exc', 'elements_inc', 
'entrytype', 'expand', 'fields', 'fracturetype', 'get_dict', 'groupid', 'hardness_max', 'hardness_min', 
'id__in', 'ima', 'ima_notes', 'ima_status', 'lustretype', 'meteoritical_code', 
'meteoritical_code_exists', 'name', 'non_utf', 'omit', 'optical2v_max', 'optical2v_min', 'opticalsign', 
'opticaltype', 'ordering', 'page', 'page_size', 'polytypeof', 'q', 'ri_max', 'ri_min', 'save', 'saveto', 
'streak', 'synid', 'tenacity', 'updated_at', 'varietyof']. Did you mean: 'elements_inc'?'''

1. Perform Detailed Queries on Geomaterials

from openmindat import GeomaterialRetriever

gr = GeomaterialRetriever()
gr.density_min(2.0).density_max(5.0).crystal_system("Hexagonal")
gr.elements_exc("Au,Ag")
gr.save()

2. Retrieve IMA-Approved Minerals

from openmindat import MineralsIMARetriever

mir = MineralsIMARetriever()
mir.fields("id,name,ima_formula,ima_year")
mir.saveto("./mindat_data", 'my_filename')

3. Search Geomaterials Using Keywords

from openmindat import GeomaterialSearchRetriever

gsr = GeomaterialSearchRetriever()
gsr.geomaterials_search("quartz, green, hexagonal")
gsr.save("filename")

# Alternatively, you can get the list object directly:
gsr = GeomaterialSearchRetriever()
gsr.geomaterials_search("ruby, red, hexagonal")
print(gsr.get_dict())

4. Retrieve Localities

from openmindat import LocalitiesRetriever

# Download Localities for certain state
lr = LocalitiesRetriever()
lr.country("USA").txt("Idaho")
lr.save()

# Alternatively, you can get the list object directly:
lr = LocalitiesRetriever()
lr.country("Canada").description("mine")
print(lr.get_dict())

5. Retrieve Type Localities for IMA-Approved Mineral Species

from openmindat import GeomaterialRetriever

gr = GeomaterialRetriever()
gr.ima(True).expand("type_localities")
gr.saveto("./mindat_data")

6. Retrieve Locality Occurrences for Single Mineral Species

Please consider using only one mineral species ID for querying localities occurrences since this query might result in many records and exceed the server limitation.

from openmindat import GeomaterialRetriever

gr = GeomaterialRetriever()
gr.expand("locality").id__in(str(id_value))
gr.saveto("./mindat_data")

Documentation and Links

• Get your Mindat API Key

• GitHub Wiki

• API Documentation

• Geomaterial Fields Definition

• R package for OpenMindat

• Built-in Help:

To explore detailed class and method documentation within the OpenMindat package, use Python's built-in help() function. This provides direct access to docstrings, showcasing usage examples and parameter details. Example:

from openmindat import GeomaterialRetriever

help(GeomaterialRetriever)

The help() is also available for the specific functions:

from openmindat import MineralsIMARetriever

help(MineralsIMARetriever.fields)

Press q to exit the help interface.

Contact Us

For further assistance or feedback, feel free to contact the development team at jiyinz@uidaho.edu.

License

Project Licence: Apache

Mindat Data License: CC BY-NC-SA 4.0 DEED

The Mindat API is currently in beta test, and while access is free for all, please note that the data provided are not yet licensed for redistribution and are for private, non-commercial use only. Once launched, data will be available under an open-access license, but please always check the terms of use of the license before reusing these data.

Authors

Jiyin Zhang, Cory Clairmont, Xiaogang Ma

Citations

If you use the data or code from this repository, please cite it as indicated below.

@misc{OpenMindat,
  author = {Jiyin Zhang and Cory Clairmont and Xiaogang Ma},
  title = {OpenMindat: A Python Package for Geomaterial Data Analysis and Retrieval from Mindat API},
  year = {2024},
  publisher = {GitHub},
  journal = {GitHub repository},
  howpublished = {\url{https://github.com/ChuBL/OpenMindat}},
  note = {Version 0.0.8}
}

Additionally, you should also reference the following paper:

Ma, X., Ralph, J., Zhang, J., Que, X., Prabhu, A., Morrison, S.M., Hazen, R.M., Wyborn, L. and Lehnert, K., 2024. OpenMindat: Open and FAIR mineralogy data from the Mindat database. Geoscience Data Journal, 11(1), pp.94-104. https://doi.org/10.1002/gdj3.204.

Acknowledgments

• This work is supported by NSF, Award #2126315.

Upgrade Logs

0.0.9

Release Date: Jun 17, 2024

• Introduced a verbose function to control the display of save notifications and the progress bar.

Usage: .verbose(FLAG) where FLAG can be:

• 0: Silent mode (no notifications)

• 1: Show save notifications only

• 2 (default): Show progress bar and notifications

• Enhanced the download logic for handling large content with oversized page sizes.

0.0.8

Released: Jun 09, 2024

• Added misspelling checks and messages for the functions in the endpoint classes. Typos in the function names will get error messages of a valid function list.
• Revised downloading logic with retries and improved stability.

0.0.7

Released: Apr 26, 2024

• The Locality country filter is fixed. The endpoint can download the data for specific countries, e.g., 'UK', 'USA', etc.

0.0.6

Released: Apr 26, 2024

• Revised a neglected get function for country endpoints.

0.0.5

Released: Apr 26, 2024

• The Internal Server Error issue in v0.0.4 is fixed from the server side.
• The get functions are now changed to get_dict.
• Added progress bars for multiple-page queries.
• Some other minor updates.

0.0.4

Released: Apr 14, 2024

• Tentative issue: Data queries involving multiple pages might return an Internal Server Error due to server-end issues. Related GitHub issue
• Added support to getting list objects of obtained data in addition to saving it to local directories.

0.0.3

Released: Apr 11, 2024

• Tentative issue: Data queries involving multiple pages might return an Internal Server Error due to server-end issues.
• Now supporting more Mindat endpoints. Not fully tested. Feedback is welcome.
• Revised API key obtaining workflow.

0.0.1

Released: Dec 14, 2023

• Initial release of the package.