Data Engineer / Data Analyst for Sports Analytics
Ilija Pavlovic
Project Description
Client Overview: The client aimed to analyze NFL 3rd down conversion percentages for all teams from 2019-2023.
Project Goals:
Extract NFL data using an authenticated API.
Clean and standardize the data.
Integrate the cleaned data into an Oracle DB using SCD Type 2.
Challenges and Solutions:
API Integration:
Challenge: Secure data extraction via an API.
Solution: Used the SportsDataIO API, scripting API calls in Python.
import requests
api_key = 'YOUR_API_KEY'
url = 'https://api.sportsdata.io/v3/nfl/stats/json/TeamGameStats/{season}'
headers = {'Ocp-Apim-Subscription-Key': api_key}
response = requests.get(url.format(season='2019'), headers=headers)
data = response.json()
Python
Proof: Captured screenshots and logs of successful requests.
Data Cleaning:
Challenge: Ensure data consistency.
Solution: Cleaned data to handle missing values, duplicates, and standardized team names.
Data Integration:
Challenge: Efficient data storage.
Solution: Developed an Oracle DB schema with staging tables using SCD Type 2.
CREATE TABLE nfl_stage (
team_name VARCHAR2(50),
third_down_conversion_pct NUMBER,
season_year NUMBER,
effective_date DATE,
expiry_date DATE,
version NUMBER,
PRIMARY KEY (team_name, season_year, version)
);
SQL
Output: The data was successfully loaded into the Oracle DB for analysis
Like this project
Posted Sep 11, 2024
The client aimed to analyze NFL 3rd down conversion percentages for all teams from 2019-2023. Project Goals: Extract NFL data using an authenticated API.
Likes
0
Views
11
