August Bulk Data: Expected Field Errors

[apeterslaw]

For using the bulk data files, I typically download the .csv.bz2 files and then use chunking combined with filtering within pandas to identify my dockets, opinions, and other items of interest. This strategy has worked well in the past, but when downloading the August files, I frequently receive an error that certain rows in the .csvs have column numbers that are different from those expected. I have attached an example when attempting to chunk and filter dockets-2024-08-31.csv

Through some ad hoc testing, my intuition is that my error mainly applies to the August file uploads, as the May 6/7, 2024 updates work fine with my approach.

Is there an update on the back end that might explain why my chunking with pandas approach is no longer helpful? And is the best solution, perhaps, to learn how to use PostgreSQL more effectively?

Thanks so much in advance for your help.

[mlissner]

Yes, we've been tinkering with the delimiters. I think you can see some changes here:

#4223
#4306

Sorry, it doesn't look like the changes made it into a release note. :(

[apeterslaw]

Thanks for the prompt reply!