EDA-Technology-Trends

Applications of various data analytics skills and techniques in data collection, data wrangling, exploratory data analysis, and creation of data visualization.

Executive Summary

Introduction

Methodology

Results -

Conclusion

Appendix

The first step in solving any data analysis problem is Data Collection. It can be done through various forms and sources. In this project, it will be collected through Web Scraping and Web APIs.

Sources including: * Surveys * Job Postings * Training portals

Data collected from the web is often if not, almost always raw and unclean. So, the data need to be prepared for the analysis.

Achieving this will be using Data Wrangling Techniques; which means that the data needs to be consistent and without missing values and errors.

After finishing up with the data cleaning, next comes the application of statistical techniques to analyze the data, and identify insights and trends. Posing questions are important to find accurate solutions, and these questions are: * What are the top programming languages in demand? * What are the top database skills in demand? * What are the highest-paying programming jobs?

Consequently, the findings will be put together by creating a Data Visualization utilizing IBM Cognos Analytics to build an interactive dashboard.

Finally, the presentation is through storytelling using a written report or as a presentation.

A global IT, and business consulting services firm needed to identify future skill requirements to keep pace with the constantly changing technologies and remain competitive.

The principle of the project is to answer one key question - What are the current skill requirements in the technology field

After going through the whole process of analyzing data, the result will inform the consulting firm of the key technical skills needed at the moment so the firm can focus on putting effort and resources into developing valuable technology assets. This is essential for the success of a business because it's only possible to understand the market's needs through knowing the current trends, otherwise, the firm might slowly slide into obscurity, as well as it will cost tons of money if the firm works in the wrong direction.

As well as it may help new graduates or any person interested in building a career based on the current industries' trends.

Data sources and methods implemented in this project are mentioned in the "project-eda-main" file.

After the data was cleaned, a dashboard was created on the IBM Cognos Analytics platform. Interactive-Dashboard

Here are some charts showing comparisons of the technology trends:

Based on "Stack Overflow Developer Survey 2019

Based on "Stack Overflow Developer Survey 2019

Javascript and HTML/CSS remain in the top two regardless of 2019 or 2020, which implies that web development is still the highest in demand. Python jumps from the top five currently to the top three in the future which implies a rapid increase in the popularity of a general-purpose programming language that's easy to learn. SQL is floating around the top three in 2019 and the top four in 202 potentially, which suggests that although it’s not at the highest ranking, it still plays a significant role in the tech field. TypeScript is not on the top five in the 2019 chart but making it to the number five in 2020 implies that it’s expected to be more popular in the future.

Based on "Stack Overflow Developer Survey 2019

MySQL was the most popular database in 2019 and still remains in the top four in 2020. Microsoft SQL is the second most popular, however, it fell out of the top five list in 2020, losing its popularity to other databases. PostgreSQL is seemingly gaining popularity, jumping from the top three in 2019 to the top one in the year 2020. It will stay competitive for quite some time. MongoDB is making its way up as well, from the top five of the list and making it to the top two in 2020.

Survey Respondent Demographic Below is a demographic dashboard that I developed based on the developer survey data. It showcases the gender, age, education level, and country of origin of the survey respondents.

Based on the "Stack Overflow Developer Survey 2019"

93.5% of the population are men, leaving only 6.5% women in the field. There is a huge gender gap in the technology workforce.

The map where it’s the darkest blue demonstrates that most of the population is heavily concentrated in the United States.

The respondent's age range is roughly in the 20s to 30s.

The education level lies mostly in the attainment of a Bachelor's degree or a Master's degree.

From the demographic data analyzed, there are large gaps in the technology industry. More opportunities can be given to women and people in other geographic locations besides the United States. so the technology workforce would be more diversified.

Based on programming languages' and databases' trends out there, and in relation to the current technical skills needed for the firm. Python, Swift, and Javascript seem to be widely used in recent years. Python is known for its readability. It allows programmers to write clear and logical codes without complexity. Swift is proposed for developing mobile applications. Javascript is exceptionally flexible as it is used for both web frontend and backend applications. They all seemingly carry the characteristics of mobility, simplicity, and flexibility. The firm can certainly focus on investing more resources in these areas.

While exploring the data, I also created a chart based on the top five most popular development environments that respondents picked. Visual Studio Code seems to be the most popular one that engineers use.

Based on the "Stack Overflow Developer Survey 2019"