Data Engineer / Data Analyst for Sports Analytics
Ilija Pavlovic
Like this project
Posted Sep 11, 2024
The client aimed to analyze NFL 3rd down conversion percentages for all teams from 2019-2023. Project Goals: Extract NFL data using an authenticated API.
Likes
0
Views
12
Project Description
Client Overview: The client aimed to analyze NFL 3rd down conversion percentages for all teams from 2019-2023.
Project Goals:
Extract NFL data using an authenticated API.
Clean and standardize the data.
Integrate the cleaned data into an Oracle DB using SCD Type 2.
Challenges and Solutions:
API Integration:
Challenge: Secure data extraction via an API.
Solution: Used the SportsDataIO API, scripting API calls in Python.
import requests
api_key = 'YOUR_API_KEY'
url = 'https://api.sportsdata.io/v3/nfl/stats/json/TeamGameStats/{season}'
headers = {'Ocp-Apim-Subscription-Key': api_key}
response = requests.get(url.format(season='2019'), headers=headers)
data = response.json()
Python
Proof: Captured screenshots and logs of successful requests.
Data Cleaning:
Challenge: Ensure data consistency.
Solution: Cleaned data to handle missing values, duplicates, and standardized team names.
Data Integration:
Challenge: Efficient data storage.
Solution: Developed an Oracle DB schema with staging tables using SCD Type 2.
CREATE TABLE nfl_stage (
team_name VARCHAR2(50),
third_down_conversion_pct NUMBER,
season_year NUMBER,
effective_date DATE,
expiry_date DATE,
version NUMBER,
PRIMARY KEY (team_name, season_year, version)
);
SQL
Output: The data was successfully loaded into the Oracle DB for analysis