Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

multiple GNSS/GPS sources support #1173

Merged
merged 45 commits into from
Apr 23, 2024
Merged

Conversation

rzinke
Copy link
Contributor

@rzinke rzinke commented Apr 1, 2024

Description of proposed changes

SUMMARY

I have redefined the GPS class to be a dedicated parent class, and have defined several child classes (e.g., UNR_GPS, ESESES_GPS) that will inherit the existing GPS processing routines. Doing so has the dual benefit of providing a straightforward structure for adding new GPS data formats, while requiring as few changes as possible to the existing MintPy code base. Related to nisar-solid/ATBD#3.

RATIONALE

The best way of implementing plugins for multiple GNSS sources is, as far as I can tell, using parent and child classes. This will provide a structured format for ingesting potential new data, maximizes reuse of existing code when new data sources are introduced, and minimizes the number of modifications to the existing code base.

The class GPS is now a parent class, which maintains the functions one would want to apply to all GPS time series, regardless of where they were processed or how they are archived. Once loaded, the three-component displacement data can be cropped, projected, displayed, and fit with mathematical models as a function of time:

# Parent class
class GPS:
    """These subroutines apply to ALL types of GPS time series.
    """
    def __init__(...)
    def open(...)
    def displacement_enu2los(...)
    def get_los_geometry(...)
    def read_gps_los_displacement(...)
    def get_gps_los_velocity(...)

The funcitons for loading and formatting the displacement time series should follow a common set of operations (e.g., load, format, etc., ...) and return the same attributes (e.g., dates, displacement components, etc.), but will obviously differ depending on the format of the source data. To account for this, one should define "child" classes, which inherit the properites and subroutines of the parent class, but for which the methods can be redefined.

# Child classes

class UNR_GPS:
    """These subroutines follow the same workflow, but work for only 
    a specific format of data UNR).
    """
    def dload_site(...):
        download UNR data
    def get_stat_lat_lon(...):
        get geographic info for UNR data
    def read_displacement(...):
	get displacement values for UNR data

# OR

class ESESES_GPS:
    """These subroutines follow the same workflow, but work for only 
    a specific format of data ESESES).
    """
    def dload_site(...):
        download ESESES data
    def get_stat_lat_lon(...):
        get geographic info for ESESES data
    def read_displacement(...):
	get displacement values for ESESES data

That way, all one would need to do to get the data into the proper format for all subsequent operations, is to define a new child class, e.g., class MyGPS with the same sequence of subroutines, rewritten for a specific data format.

Once a child class has been defined, it can easily be recalled by the parent class using the static method GPS.get_gps_obj_by_source(<processing_source>). This offers both convenience and structure, and reduces the number of if statements in the broader code base. For example

from mintpy.objects import gps
gps_source = 'UNR'
GPS = gps.GPS.get_gps_obj_by_source(gps_source)

is all one needs to add in order to not break the existing libraries.

Reminders

  • Pass Pre-commit check (green)
  • Pass Codacy code review (green)
  • Pass Circle CI test (green)
  • Make sure that your code follows our style. Use the other functions/files as a basis.
  • If modifying functionality, describe changes to function behavior and arguments in a comment below the function declaration.
  • If adding new functionality, add a detailed description to the documentation and/or an example.

New functionality

The user can now specify GPS processing source. If not specified, the default source is UNR NGL -- the only source supported currently -- and all functions can be used as before, as if no changes were made.

@rzinke
Copy link
Contributor Author

rzinke commented Apr 1, 2024

pre-commit.ci autofix

@rzinke
Copy link
Contributor Author

rzinke commented Apr 1, 2024

pre-commit.ci autofix

@rzinke rzinke marked this pull request as ready for review April 3, 2024 02:01
@rzinke rzinke mentioned this pull request Apr 3, 2024
2 tasks
@yunjunz yunjunz self-requested a review April 3, 2024 15:28
@yunjunz
Copy link
Member

yunjunz commented Apr 3, 2024

I always wanted to rename the GPS naming into GNSS, for a more generic purpose. This includes all the related functions, variables, and option names, and code comments/documentation. As for the view.py options, we could support both gps and gnss for back-ward compatibility. @rzinke Shall we do it in this PR?

@rzinke
Copy link
Contributor Author

rzinke commented Apr 3, 2024

I always wanted to rename the GPS naming into GNSS, for a more generic purpose. This includes all the related functions, variables, and option names, and code comments/documentation. As for the view.py options, we could support both gps and gnss for back-ward compatibility. @rzinke Shall we do it in this PR?

Sure! And I agree philosophically.

@yunjunz
Copy link
Member

yunjunz commented Apr 3, 2024

@rzinke Could you go for this change?

I will check more at the PR later this week.

@rzinke
Copy link
Contributor Author

rzinke commented Apr 3, 2024

@rzinke Could you go for this change?

I will check more at the PR later this week.

Yes. I am working on it right now. Would you like the whole module gps.py renamed to gnss.py?

@yunjunz
Copy link
Member

yunjunz commented Apr 3, 2024

Yes please.

@rzinke
Copy link
Contributor Author

rzinke commented Apr 3, 2024

pre-commit.ci autofix

@rzinke
Copy link
Contributor Author

rzinke commented Apr 3, 2024

pre-commit.ci autofix

@yunjunz yunjunz changed the title GNSS refactoring: parent/child class multi-source GNSS solution support Apr 15, 2024
@yunjunz yunjunz changed the title multi-source GNSS solution support multiple GNSS/GPS sources support Apr 15, 2024
rzinke added 2 commits April 15, 2024 14:55
…efault. Changed Generic to upper case. Added options for JPL-SIDESHOW and GENERIC in arg_utils choices.
@rzinke
Copy link
Contributor Author

rzinke commented Apr 15, 2024

Hi @yunjunz, thanks for the feedback. I have incorporated the proposed changes into the gnss.py module. Please have a look when you get the chance. Cheers.

yunjunz and others added 19 commits April 19, 2024 00:43
+ read_JPL_SIDESHOW_station_list(): use `np.loadtxt` for simplicity

+ search_gnss(): move crop in space/time/# code into here, for simplicity and readability
+ objects.gnss:
   - search_gnss(): set back the min_num_solution default value to 50
   - use os.path.isfile() to check the existence of file, instead of os.path.exists()
   - rename get_gnss_los_obs() to get_los_obs()
   - simplify GNSS.__crop_to_date_range__() and display_data()
   - rename display_data() to plot()
   - call self.open() when needed in the member functions, so that it can be removed on the user side, to simplify the usage
   - fix two issues: 1) add __init__() in the child class, define self.file; 2) retry downloading if failed; 3) retry downloading if loadtxt() failed on existing data file, due to download interuption in the previous run
   - rename child class names, to be consistent with the folder name

+ utils.plot.plot_gnss(): print the nearest GNSS site to the current reference point, to faciltate the --ref-gnss option setup
+ rename get_stat_lat_lon() to get_site_lat_lon()

+ rename read_*_station_list() to read_*_site_list()
+ objects.gnss.py
   - get_los_obs(): use different CSV file name for different sources
   - add get_ESESES_url_prefix() to speedup the ESESES downloading / calculation
   - rename file_url to url for member var
   - add a new member var: url_prefix
   - move dload_site() to the parent class for better reuse.

+ utils.plot.plot_gnss(): specify the missing gnss_source arg

+ utils.ptime.get_date_str_format(): add YYYYMMDD:HHMMSS format
Copy link
Member

@yunjunz yunjunz left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Hi @rzinke, I finished the review of this long PR finally. Everything looks good to me now, except for one last comment below, regarding the plotting code below within GNSS.displacement_enu2los().

Let's ignore the Codacy warning: it's fine.

src/mintpy/objects/gnss.py Outdated Show resolved Hide resolved
+ displacement_enu2los(): remove the plotting code

+ move the matplotlib import into the sub-function, to speedup the warmup proc
@yunjunz yunjunz merged commit 4396cb6 into insarlab:main Apr 23, 2024
7 of 8 checks passed
@rzinke rzinke deleted the gps_object_split branch July 25, 2024 18:34
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
None yet
Projects
None yet
Development

Successfully merging this pull request may close these issues.

2 participants