Capstone Project Week 9

 What did you accomplish this week?

This week I worked on updating our NHC Recon Dropsonde parsing library with the capability to gather individual dropsonde files by a URL from the online NHC archive web server, gather all dropsonde files using links scraped from an NHC archive page, and integrated pypgstac with psycopg2 to handle inserting generated STAC items into a dropsonde collection and into a PgSTAC database. To familiarize myself with our system and determine the best path forward for deployment, I setup a development environment on my local machine running docker containers for stac-fastapi-pgstac with our parsing package installed on it, stac-db for the Postgres database, and stac-browser to test a web based GUI browser. I made modifications to the fastapi server within the stac-fastapi-container, adding endpoints to post dropsonde file and archive URLs for processing. Following those modifications I ran a manual system test resulting in hundreds of dropsonde observations being parsed and loaded into a STAC Catalog as demonstrated in the images below.

 

 

What is your plan for next week?

 Next week I will continue testing the containers and tie them together with docker compose. In addition, I will make updates to the parser to cut back on extraneous data within the STAC items, as well as do additional parsing to convert desired properties into a more human readable format.

 What challenges, if any, are you currently facing in project development? Do you need instructor assistance? 

 So far I'm not facing any major issues, however development has taken longer in some areas due to lack of detailed documentation for certain libraries and the necessity to reverse engineer some functionality.  

Comments

Post a Comment

Popular posts from this blog

Software Engineering Week 4

    The most interesting thing I have learned about in my reading of "Software Engineering at Google" are how each of the various processes are executed. While I am familiar with most of the processes I've read about so far, including code reviews, unit testing and system testing, the manners in which these are executed at Google are different from how they have been executed in my professional experience. For code reviews in particular, Google engineers ensure they invite various other software engineers to fill the roles of correctness and comprehension checking, language readability checking, and code owner. However, in my experiences this is not explicitly required. While a single engineer on the team often fills a couple of these roles, there wasn't an engineer specifically dedicated to carry out each of these tasks. Another interesting thing I learned was a recommendation about the use of the "sleep()" function in testing. While sometimes waiting is re...
Read more

Capstone Project Week 10

   What did you accomplish this week? This week I looked into integrating the stac-browser container within the docker compose YAML file provided in the stac-fastapi-pgstac project. In addition I cleaned up some unnecessary data included in STAC items written by our parser as well as read further documentation from the National Hurricane Center on how I can do additional parsing on dropsonde remark messages. I also published a build and tested docker image including our team's parsing tools to docker hub to allow it to be pulled from our AWS instance. What is your plan for next week? Next week I will work with my team to get our STAC Server running and hosted within an AWS server for demonstration and testing with our stakeholders.  What challenges, if any, are you currently facing in project development? Do you need instructor assistance?  So far local development has been smooth on a Windows 11 operating system. Containers are able to be built and ran using docker...
Read more

Capstone Project Week 11

What did you accomplish this week? This week I pushed updates to our parser development branch which included additional parsing for remarks and removal of certain metadata properties that were unnecessary for describing the who, what, when, and where of a dropsonde message. In addition I was able to get our parser, STAC API, and STAC browser containers running and hosted AWS, load our STAC database with STAC items using our custom API endpoint integrated with our parser, and test our application's availability on the internet. After confirming our application's accessibility, I scheduled a demonstration with our mentor to show our progress and attain feedback. What is your plan for next week? Next week I will demonstrate our STAC parsing and catalog browsing capabilities within our container hosted on AWS to our project mentor.   What challenges, if any, are you currently facing in project development? Do you need instructor assistance?  I'm not facing any challenges at...
Read more