Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

add more WQ data ts plots #2

Open
dotis opened this issue Jun 29, 2023 · 5 comments
Open

add more WQ data ts plots #2

dotis opened this issue Jun 29, 2023 · 5 comments
Labels
enhancement New feature or request

Comments

@dotis
Copy link
Collaborator

dotis commented Jun 29, 2023

@7yl4r There are two main formats for the WQ data. There is a .csv file for each param (chl, turb, nitrogen, silica, etc.) or there is a merged .csv with all of them in the same file. I'm not sure which will be easier to ingest by this tool. The merged file is fairly large (~64MB, ~600,000 rows).

@dotis
Copy link
Collaborator Author

dotis commented Jun 29, 2023

Looking at the merged file, I see some differences and inconsistencies with program names, so I think it will be best to use the individual .csv files for each parameter.

@7yl4r 7yl4r changed the title Format for WQ data add more WQ data ts plots Jul 4, 2023
@7yl4r
Copy link
Member

7yl4r commented Jul 4, 2023

@dotis : I tried adding additional plots but ran into an issue. I think the issue is that some of the wq parameters may not be coming from the same stations. This could be the same name inconsistencies you were seeing in the other file.

Here is an example line from data/Merged_turb.csv for which there is no station information:

Miami Beach,35,12,2022,Turb,1.87,NTU,,

We can add more stations to the map but we will need details for those stations added to station_sampling_periods_for_all_programs.csv.

@dotis
Copy link
Collaborator Author

dotis commented Jul 6, 2023

Looking at the merged file with all wq params, I see that this is more comprehensive than the individual param files and includes lat/lon for each site. Perhaps it is actually better to use this file. I will ask Alex Fine at AOML.

@7yl4r can we ingest from a large file (~64MB)? Currently, it is in .csv format. Or, do we need to break it up into smaller chunks?

@7yl4r
Copy link
Member

7yl4r commented Jul 6, 2023

<100MB is acceptable, <50MB is preferred. Drop it in https://github.com/7yl4r/wq-dash/tree/master/data and we'll see.

@dotis
Copy link
Collaborator Author

dotis commented Jul 6, 2023

GH says, "No".

"Yowza, that’s a big file. Try again with a file smaller than 25MB."

@7yl4r 7yl4r added the enhancement New feature or request label Oct 25, 2023
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
enhancement New feature or request
Projects
None yet
Development

No branches or pull requests

2 participants