Data Engineer / Data Analyst for Sports Analytics

Project Description

Client Overview: The client aimed to analyze NFL 3rd down conversion percentages for all teams from 2019-2023.

Project Goals:

Extract NFL data using an authenticated API.

Clean and standardize the data.

Integrate the cleaned data into an Oracle DB using SCD Type 2.

Challenges and Solutions:

API Integration:

Challenge: Secure data extraction via an API.

Solution: Used the SportsDataIO API, scripting API calls in Python.

import requests

api_key = 'YOUR_API_KEY'

url = 'https://api.sportsdata.io/v3/nfl/stats/json/TeamGameStats/{season}'

headers = {'Ocp-Apim-Subscription-Key': api_key}

response = requests.get(url.format(season='2019'), headers=headers)

data = response.json()

Python

Proof: Captured screenshots and logs of successful requests.

Data Cleaning:

Challenge: Ensure data consistency.

Solution: Cleaned data to handle missing values, duplicates, and standardized team names.

Data Integration:

Challenge: Efficient data storage.

Solution: Developed an Oracle DB schema with staging tables using SCD Type 2.

CREATE TABLE nfl_stage (

team_name VARCHAR2(50),

third_down_conversion_pct NUMBER,

season_year NUMBER,

effective_date DATE,

expiry_date DATE,

version NUMBER,

PRIMARY KEY (team_name, season_year, version)

);

SQL

Output: The data was successfully loaded into the Oracle DB for analysis