Skip to content

Latest commit

 

History

History
214 lines (167 loc) · 6.05 KB

README.md

File metadata and controls

214 lines (167 loc) · 6.05 KB
Raw

Senior Data Engineer

San Diego, CA

@dlubawy

Experience

Vocational

Nov 2022–May 2024
Senior Data Engineer
Cue Health
San Diego, CA

  • Led the design and implementation of a custom data pipeline using Go to extract, store, and transform real-time manufacturing data, handling 15 million images/month on the order of petabytes per year in total.
    • Developed a custom caching system across goroutines called through Cloud Run to ensure minimal query counts within BigQuery. Reduced image indexing costs by 94% versus a non-cached system.
  • Led the data engineering team in implementing a system for collecting, processing, and presenting manufacturing data. This provided the first automated calculation of overall equipment effectiveness (OEE) at Cue. It improved new product development, field failure analysis, and manufacturing efficiency.
  • Took over as lead software engineer for the internal cartridge application built with TypeScript. Developed changes to implement reporting on multiplex tests for internal teams.
  • Designed an improved second iteration on the first data lake architecture for Cue that emphasizes self-service approaches and an analytics engineering focus using Dagster, dbt, BigQuery, and Looker.

Mar 2022–Oct 2022
Data Engineer
Cue Health
San Diego, CA

  • Created a custom server-less data pipeline in Python for streaming billions of rows from noncompliant CSV files into a BigQuery table.
  • My first architecture presentation set the standard and was described by the chief architect as “the best and most comprehensive.”
  • Lead engineer on migrating the Cue marketing team from the Salesforce Datorama platform to an in-house data lake approach with BigQuery and Looker. Moved 7 marketing connectors over from Datorama to the new architecture.
  • Designed and built Cue's first data lake platform using services and tools such as Fivetran, BigQuery, dbt, and Looker.
    • Took the data platform from 2 models to a large DAG of over 1,500 SQL transformation models.
    • Created a data lake that could account for GDPR, CCPA, HIPAA, and SOX requirements.
    • Designed the data lake to work across cloud regions (USA, Canada, and Singapore).

Jan 2019–Mar 2022
Software Developer (Contractor)
National Aeronautics and Space Administration (NASA)
Hampton, VA

  • Support the Committee on Earth Observation Satellites (CEOS) Systems Engineering Office (SEO) under the direction of Dr. Brian Killough to further NASA's goals in the Earth observation community.
  • Lead developer on a project in partnership between CEOS and Google Earth Engine (GEE) team to promote the use of Earth Engine data with the Open Data Cube (ODC) project.
    • Developed a novel interface to the ODC using GEE's REST API and Python libraries.
    • Created a sandbox environment using the ODC in Google Colab to facilitate collaboration and reduce costs by 95% (https://github.com/ceos-seo/odc-colab).
  • Lead developer on building a training website using Django for NASA's Working Group for Capacity Building and Data Democracy team in CEOS (https://training.ceos.org).
  • Support the CEOS initiative and Earth observation community by creating various Jupyter notebooks to serve as examples and to explore new algorithms.
    • Made a PoC for water detection using auto-threshold techniques on Sentinel-1 SAR data.
    • Implemented a notebook for detecting mangroves from Landsat imagery which helped countries in Africa understand their coastline inundation.
    • Used the random forest machine learning algorithm to create a notebook for land classification to help provide insights into UN Sustainable Development Goals (SDGs).
  • Lead developer on creating inventory web applications for CEOS using Django (https://ceos.org/data-tools/).
  • Coauthored multiple conference papers for CEOS, and presented a paper at IEEE's IGARSS conference in 2020 (https://ieeexplore.ieee.org/author/37088754294).

Dec 2018–Mar 2022
Software Developer
Analytical Mechanics Associates (AMA)
Hampton, VA

  • Provided software development support such as data analysis, web development, and backend infrastructure to NASA under AMA's TEAMS 3 primary contract.
  • Full stack developer on NASA subcontract to develop a real-time telemetry web application for the Lunar VIPER mission. This was to provide support to public scientific endeavors and promote public engagement with the mission.
  • Built a React Native Android application to act as a UI to the ODC.

Dec 2018–Jun 2019
Software Developer
Insight Global
Hampton, VA

  • Contractor for AMA and subcontractor for NASA. Converted to full-time at AMA after 6-month contract ended.

Jan 2018–May 2018
Student Associate
Miami University
Oxford, OH

  • Helped Dr. Donald Ucci develop a lab for an RF course at Miami University.

Sep 2016–Apr 2019
Software Developer
Valbridge Property Advisors
Las Vegas, NV

  • Created and maintained a Python web application through Django to generate reports using ArcGIS from ESRI.

Miscellaneous

2021
Scientific Committee Member
IEEE International Geoscience and Remote Sensing Symposium (IGARSS)

  • Reviewed research papers for approval to the IGARSS conference.

Education

2018
B.S. Electrical Engineering
Miami University
Oxford, OH
Concentration in Computer Systems

Computer Languages

Python
Go
SQL
JavaScript/TypeScript
Bash
C/C++
Rust
Nix

Engineering Skills

Data Engineering

  • BigQuery
  • dbt
  • Fivetran
  • Dagster

Web Frameworks

  • Django
  • Flask
  • Vue.js

Cloud Platforms

  • GCP
  • Firebase
  • AWS
  • DoiT

Infrastructure

  • Docker
  • Kubernetes
  • Terraform

Data Science

  • Jupyter
  • NumPy/SciPy
  • Pandas

Electrical Engineering

  • MATLAB
  • Altium
  • SPICE

Software Engineering

  • Git
  • GNU tools
  • Linux

Mobile Development

  • React Native

https://github.com/dlubawy | https://github.com/dlubawy-ama | https://www.linkedin.com/in/andrewlubawy