Madhubala Gnanamoorthy describes her typical work-day as a data engineer at Crayon, shares some of her favorite data tools to work with and what Crayon value speaks to her the most.
My workday usually starts at 9 AM. I catch up with my team to understand the priority tasks, and make sure the most critical work gets done first. We share responsibility for maintaining both production tasks and jobs which generate critical reports that are shared to clients. We make sure they’re running correctly and carry out any fixes if needed.
A greater part of my day is overtaken by new development, building data pipelines and automating reports shared to different client teams.
I try to make sure that all the work assigned to me is moving forward without any glitches.
On tech and tools
To be a strong data engineer, a diverse skill set is required. Proficiency in different tech will help build a distributed system, create dependable pipelines and combine different data sources effectively. I am proficient in
- AWS & Apache Services like Spark, Hive, EMR, S3, Airflow
- Shell scripting, Cronjobs
- Python/Scala/Java
- Database fundamentals like SQL
- Relational DB’s like Mysql/PostgreSQL
At Crayon, we have been using AWS Spark for processing our data, almost in real-time. Recently, our team moved to AWS Glue to leverage the same data processing.
AWS Glue is useful when we have flexible requirements, and as it is server-less, we do not need to configure and manage any computing resources. It simply helps crawl, discover and organize data and prepare it for analytics.
On her favorite Crayon Value
Crayon values drive the attitude and behaviors that company wants to see within us. I really like the way how each value has been set. My favorite is Be the Benchmark.
I believe that one can master this value only if there is personal involvement and alignment with the work they do. This will eventually help the individual as well as the company to reach excellence.
Are you outside the Crayon Box and ready to jump right in? We’re looking for innovative, quirky and result-driven folk to join the Talented Tangram fam! Join the box today.
