v0.1.8

Fixed a long-standing performance issue with loading the python module. Imports are now done on demand, shaving ~4 seconds off loading the library (including starting the server or running any of the scripts).

Improved ingest flow #34 implemented. When data is ingested we now:

• Check types of columns and generate summary statistics. this will enable future UI
• Check for array columns and suggest importing those as embeddings
• Check for name collisions when uploading a dataset, giving a warning if a collision is detected

0.1.7 Setup process improvements

• A number of styling improvements are made to the setup page.
• Fixed an annoying loading issue that was adding ~4 seconds to each part of the process.
• Added nltk top words as a CPU friendly way to summarize clusters

0.1.6 Export data

Adds an export page as described in #32

This makes it easy to download the data associated with a scope. Existing scopes will need to be updated to see the scopes.parquet which can quickly be done from the Setup page for a scope and clicking the "Overwrite" button. This creates the new combined file of all the scope's information for convenient use outside this library.

0.1.5 Python packaging fix

Improved build process fixes an issue with stale web code being bundled.

Settings page in UI

Adds a Settings page where you can see the current data directory and .env file locations. It also shows which API keys have been set and allows you to securely set new API keys for each of the supported services.

0.1.3 Import embeddings

Import existing embeddings with the import_embeddings function #13

0.1.2 Aligned UMAPs

This release enables aligning UMAPs via the UI. It also supports Matroyshka embedding models, allowing you to make smaller embeddings from existing embeddings without needing to recompute them.

• Closed Aligned UMAPS issue #25
• added cluster selection epsilon parameter to clustering step
  • https://hdbscan.readthedocs.io/en/latest/how_to_use_epsilon.html
• truncate existing matroyshka embeddings
• start adding tooltip info to setup UI

0.1.1 Small fixes

• matroyshka embeddings #27
• layout fixes #5 #16
• openai gpt4 model list fix #18
• transitioning visualizations when changing umaps and scopes

Initial release

The initial release of Latent Scope. Supports ingesting, embedding, mapping, clustering and exploring unstructured text data.