Sowmya Lanka
Data Scientist | ML, NLP & GenAI Solutions
- 1
- Followers
Cleaning text data is 80% of any NLP project — here's my actual checklist:
Strip HTML, hashtags, special characters
Lowercase + tokenize
Remove stop words (carefully — sometimes they matter)
Lemmatize, don't just stem, if word meaning matters
Most "bad model" complaints I've seen are actually bad-data problems.
3 things I check before building any prediction model:
Multicollinearity — are my features actually independent, or just noise duplicating each other?
Outliers — one bad data point can quietly wreck your regression line.
What the client actually needs — R² of 0.95 means nothing if...
What I built
Developed a machine learning solution to predict Buy Box and sales prices using historical analytical data and Linear Regression.
How it works
Analyzed the data for linear relationships, autocorrelation, multicollinearity, homoscedasticity, and normally distributed...
What I built
Developed a Natural Language Processing (NLP) solution to detect DI Flag issues by analyzing and classifying text descriptions.
How it works
Cleaned text data by removing hashtags, HTML tags, special characters, and numeric values.
Applied text preprocessing techniques...
What I built
Developed an automated clinical text summarization solution using Google Gemini AI and prompt engineering to generate concise summaries from lengthy clinical chart notes, transcripts, and unstructured medical text.
How it works
Processed lengthy clinical notes and...
