Morgenland Teppich - Ecommerce Site
Nirjal Paudel
Morgenland Teppiche Germany
I worked as a backend developer here (40 hours/week). I was assigned various works and did independent research to fix the company day to day workflow and improve the company's daily workflow and implemented cloud computing with AWS.
Before Me [Non-Technical Preseptive]
The entire company process was extremely slow.
Everything was handled in excel files. 20GB of excel file
The website was static
Backend was poor like US and UK had the same currency rate
Sync process that was often dragging the sales process down
No user behaviour study
Little and outdated analysis of sales and orders
Very poor traffic handling capacity
Poor Cross selling strategy and its implementation
Poor and slow search and filters features
Poor organization documentation and knowledge base
After a year of my contributions
Non Technical
After a year I managed to solve many of their problem. They just had the best off season for their standard and as of the time of this writing. I think they are on their way to have their best seasons. We are often using data driven descision and making them to move their descision making towards data took a lot of persuading.
Here is how we have changed
3 seperate databases now.
Search and filter database in PostgreSQL
User tracking database in PostgreSQL
Organization inventory and for user analysis in MySQL
Organization Knowledge Base is now setup in Notion
User behaviour analysis is now implemented in the backend
Rest API and dashboard is implemented to empower data driven descision.
Cross sellings are now done based on user cart analysis and by using Machine Learning algorithms
Search and filters are now based on latest research on information retrieval systems and full text search with very high throughput.
Automation: Daily and frequently repeating tasks are now autmated
We setup cloud based solution in AWS for traffic related problems.
Technical
Seperate Database for each usecase: After careful inspection, our access pattern included the three following kind
Transactional and Analytical We built the transactional database which handled search in Postgres
2. Analytical database handled all our calculations in Mysql.
3. User Behaviour Tracking: For user behaviour tracking and machine learning
2. Organizational Knowledge Base: Knowledge base for organizational knowledge was implemented in Notion which utilizes PARA method
3. Rest API and dashboard: We made rest api using Django, Django Rest Framework, and also utilized celery and flower for background task processing and automation needs.
4. Cross Selling: Now powered by user behvaviour. We utilized matrix factorization and shopping cart analysis with both explicit and implicit feedbacks.
5. Search and filters: Search and filters are now 100 times fast from 6s to 0.6s in P99. [Nginx logs report]. We also used postgres's full text search, reduced joins needed and utilized JSONB columns heavily.
6. Automation: Celery was used for automation pruposes and we made views for automating manual tasks
7. Insights Collection: We are now collecting insights from user behaviours on where can we improve our server. I did SQL aggregations and implemented logs parsing tricks to find opporunties
8. AWS: Now the whole infrastructure is build on AWS and we have build a wrapper server around the old CRM system extending its features. The system is there now as well but not much of pain right now. We use
i. S3
ii. Ec2
iii. Lightsail
iv. DynamoDB
v. RDS
vi. Aurora
Technical Learnings from Morgenland
Here are some of technical learning I gained during my time in Morgenland
I am the now the lead backend developer here.
