Error when trying to upgrade my langfuse deployment

[nithin1995]

Hi Team, when I try to upgrade langfuse, my datasets runs are not getting loaded. i have it deployed on k8s cluster. Right now my deployment config points to a particular version of langfuse 2.78.0. To upgrade i edited my deployment config and made the version as 2, so it should pull the latest image.

The webserver is getting upgraded to 2.81.0 but the dataset runs are not getting loaded on UI. so i reverted my deployment config back to 2.78.0. Thankfully did not lose any data and its working.

Can you help me how to upgrade this?

[dosubot[bot]]

When upgrading from Langfuse version 2.78.0 to 2.81.0, there are some changes that could affect dataset runs. Specifically, there is a change in the database schema where a new column named "metadata" of type JSONB has been added to the "dataset_runs" table. Additionally, the API has been extended with new endpoints for programmatic management of datasets ^[1][2].

To resolve the issue with dataset runs not loading, ensure that your database schema is updated to include these changes. You might need to run the necessary database migrations to add the new "metadata" column. Also, verify that your application is correctly configured to use the new API endpoints if applicable.

If you have any custom configurations or scripts, make sure they are compatible with the new version's changes. After making these adjustments, try upgrading again to see if the issue persists.

To continue talking to Dosu, mention @dosu.

[nithin1995]

Ideally the database migrations must be applied automatically while upgrading the langfuse version right?

[maxdeichmann]

Yes, they are applied automatically for you.

[maxdeichmann]

@nithin1995 when upgrading, did you see any logs on the server? This would help me understand what is going on.

[maxdeichmann]

I would remove SHADOW_DATABASE_URL as well. I assume you are NOT connecting via pgbouncer or any other proxy?

Before you proceed with the upgrade:

• please ensure you have backups for the database enabled in case something goes wrong
• I am still not sure what happened but proceeding here seems to be the only way to get new insights. Hence not sure whether this will work.
• Did you manage to reproduce the behavior in any other environment?

[nithin1995]

1. I have a pg_dump of the entire database taken yesterday. I have backup.
2. Sure, let me try out and tell you how it went.
3. Nope

I will remove SHADOW_DATABASE_URL, DIRECT_URL. I will change LANGFUSE_AUTO_POSTGRES_MIGRATION_DISABLED to "false" and change the langfuse version from 2.78.0 to 2 and redeploy.

[maxdeichmann]

Ok, let me know how it went.

[nithin1995]

Hi @maxdeichmann , I made the upgrade just now. Looks like its working in first glance. I will test more deeply tomorrow and raise in case something is broken. Thanks for all your help !!

[maxdeichmann]

Thanks a lot! Feel free to reopen this in cas you tun into issues