Onboarding

This page contains the onboarding guide to Whyphi’s applications.

Zap

Setting Up - Local Environment

Zap is developed using Python 3.9. Ensure that you have Python 3.9 installed. To manage dependencies, Zap uses Pipenv, a Python virtualenv management tool.

Install pipenv to your machine by using: pip install --user pipenv or by following the pipenv documentation.

Setting Up - Amazon Web Services

Zap utilizes many AWS services. To get started with AWS, ensure that you have the AWS CLI installed. Then, set the necessary AWS configuration within your system using: aws configure. Receive the necessary AWS credentials from another member in the PCT Tech Team.

This is how your aws configure should look like:

AWS Access Key ID [********************]: {AWS Access Key}
AWS Secret Access Key [********************]: {AWS Secret Access Key}
Default region name [us-east-1]: us-east-1
Default output format [json]: json

Local Development

To turn on the virtual environment using pipenv:

pipenv shell

To install all necessary dependencies within pipenv from our repo:

pipenv install

To install any additional dependencies within pipenv:

pipenv install {dependency name}

To enable local server for Chalice:

chalice local

Exploring & Testing API Endpoints: Tools & Tips

Calling APIs efficiently is essential for developers. While cURL requests are powerful, there are user-friendly alternatives:

1. Postman
2. HTTPie

Choose a tool that suits your comfort level and workflow. Both Postman and HTTPie streamline API testing and development tasks.

Directory Structure

In AWS Chalice, the application itself is served under the root directory in a file called app.py. All other subdirectories and modules need to be created under a directory called chalicelib.

Exploring chalicelib

Within the chalicelib directory, many directories exist. Let’s now see the purpose of each directories:

chalicelib/api

This is where all API endpoints are created in our backend application. We leverage blueprints to organize our application into logical components.

To create your own blueprint and API endpoints, create a new file in api, intialize a new blueprint object, and create your endpoints:

To ensure that your API gets registered to the chalice application, go to app.py, import the blueprint and register it on app object:

chalicelib/models

Models is where we create the data structure and types for data that the application works with. To ensure that our data is consistent throughout our application, we utilize Pydantic, a data validation library for Python.

If there is a need to create new data types for new features, initialize a new model through Pydantic, and you can use it for validation purposes.

chalicelib/services

For Zap, we attempt to use a layered architecture. The general schema of layers on a backend application is shown below:

The services directory acts as the service layer for Whyphi, in which it contains business logic and handles data manipulations. “Generally services contain information that is related to their domain.

For example, if we have “Mail Service” we expect that sending/receiving emails happens there like in a real life. The same is fair for the codebase. Services (and their methods) handle the business logic which means that they are responsible for transforming data, performing additional actions (like asking the repository for additional data or another service for processing some logic for it).”

To learn more: https://dev.to/blindkai/backend-layered-architecture-514h

To create a new service module, create a new file called {service_name}Service.py and create a class. Then, create an instance of that class at the bottom of the file, so we can import the instance directly from other files. Here is an example of ApplicantService.py:

# chalicelib/services/ApplicantService.py

class ApplicantService:
    def __init__(self):
        self.table_name = "apps"

    def get(self, id: str):
        data = db.get_item(table_name=self.table_name, key={"applicantId": id})
        return data

    def get_all(self):
        data = db.get_all(table_name=self.table_name)
        return data

    def get_all_from_listing(self, id: str):
        data = db.get_applicants(table_name=self.table_name, listing_id=id)
        return data

applicant_service = ApplicantService()

chalicelib/modules

For Whyphi, we take advantage of a lot of microservices and cloud applications. This means that we need different functionalities for each service to handle the data and/or logic. Thus, for each microservice or application, we would create a file and put them in modules.

For example, in Whyphi, we use MongoDB and AWS S3. Each microservice would have its own file with specific functionalities that we would use. For those services, we would create the following files:

• chalicelib/modules/s3.py
• chalicelib/modules/mongodb.py

Unit Tests

Unit testing is an important part to ensure that changes on our codebase does not break existing features and functionalities. For Zap, we mainly use pytest, a Python testing framework.

Creating Test File

All unit tests are created under the tests directory, and each directory and/or file would correspond to each file in chalicelib and the application itself. For example, if we wanted to create unit tests for chalicelib/services/ApplicationService.py, we would create a file like tests/services/test_application_service.py.

Creating and Running Tests

NOTE: All test files names must have the prefix test_

To create a test, create a function with the prefix name test_, and proceed with unit testing.

To run the unit test, run the pytest command and it will show if you passed or failed any tests.

Code Coverage

Code coverage is a metric that can help you understand how much of your source is tested. It’s a very useful metric that can help you assess the quality of your test suite.

To run and generate the code coverage report, run make coverage. You will have re-run this command everytime you make changes to your unit test suite.

To view the code coverage report, open htmlcov/index.html and it will give a summary of how much code was covered from the unit tests. Click on a specific file from the report to get a detailed view of which lines were covered and not.

The green lines indicate that the unit tests hit these lines, where as the red lines mean the opposite. For this example, we can see that the else case and the exception case on delete_item was not tested in the db.py file:

Setting Environment Variables in Chalice

Setting up environment variables are a little bit more complicated in Chalice. However, it is crucial to understand and know how to implement them as we work with sensitive credentials from other external services.

Currently, our local backend is configured to utilize AWS services on a development environment/service. This means local environment variables are technically hosted on the cloud as well.

Creating an Environment Variable

First, navigate to AWS Systems Manager. Then, navigate to Parameter Store on the sidebar under Application Management:

Then, click Create parameter to create an environment variable:

Fill out all the necessary details for your environment variable (try your best to keep the name formatting relevant and consistent):

Click Create parameter.

Providing Environment Variable Access for Chalice

To use AWS services on chalice, the necessary permissions need to be provided or else AWS denies the specific resource when requested. Param Store environment variables are considered as a resource and we need to specify that our application should have access to request it.

On Zap’s directory, navigate to .chalice. Inside it, you will see both policy-dev.json and policy-prod.json. Inside both json files, navigate to "Resource" and add your new parameter to the respective files. Your Chalice application should now have access to request the new environment variables you created.

Portal

Node.js & Prerequisites

To get started with Portal, ensure that you have Node.js downloaded and your node version is v20.x.x.

Before you start working with this project, make sure you have the following prerequisites in place:

• Node.js: Ensure that you have Node.js installed. If not, you can download it from here. Verify your Node.js version by running the following command:

  node -v

• Your Node.js version should be at least v20.x.x. If not, consider updating Node.js using the following command:

  npm install -g n

Getting Started

To install all necessary dependencies to run the project:

npm install

To start the development server, run the following command:

npm run dev

Open http://localhost:3000 with your browser to see the result.

.env.local

Since Portal uses a lot of different services throughout different features and functionality, there are lot of sensitive information and data that needs to be used. For our application, we handle environment variables through a file called .env.local. Create this file on the root of the project.

Once you created .env.local, get the necessary credentials from another member in the team and copy and paste it. If the project is still not working with the credentials, restart your application.

.env.local should look something similar to this:

NEXT_PUBLIC_API_BASE_URL=http://xxxxxxxxxxxxxxxxx
GOOGLE_CLIENT_ID=xxxxxxxxxxxxxxxxx
GOOGLE_CLIENT_SECRET=xxxxxxxxxxxxxxxxx
MONGO_USER=xxxxxxxxxxxxxxxxx
MONGO_PASSWORD=xxxxxxxxxxxxxxxxx
NEXTAUTH_SECRET=xxxxxxxxxxxxxxxxx
NEXTAUTH_URL=http://xxxxxxxxxxxxxxxxx

...

Directory Structure

WORK IN PROGRESS

Tracker

Tracker is Whyphi’s progress tracking repository. The repository itself does not contain any code, but rather utilizes GitHub’s issues feature to track work in progress and features that are complete.

Why Tracker? It’s important to gain visibility on which features are completed before a version release to production. Since our backend and frontend repositories are separated, there needs to be more attention taken care of before deploying our services to production. Tracker helps minimize the risk of bugs and issues for a decoupled service.

How to use Tracker?

All progress will be located on issues: https://github.com/whyphi/tracker/issues

If you are currently working on a feature, copy and paste the link to your Pull Request next to the checkbox:

1. Click on Edit

2. Copy and paste your pull request URL next to the checkbox for the corresponding feature

3. Click the Update comment button

4. If you complete your feature, mark the checkbox to indicate your feature has been completed.