Senior Technical Data Engineering Lead | Paytm
9 Years of Experience
Gurugram, Haryana, India
-
-
Not Available
Enthusiastic Data Engineering Lead (9+ years of experience) with background of working proactively in dynamic environments and domains. Strong experience in Big Data Ecosystem, designing and developing pipelines and architectures. Have an experience of handling large teams end to end. Core competencies include Big Data Engineering and Analytics, Design and Development Engineering, Enterprise Data Warehousing Solutions with strong communication and team handling soft skills.
Paytm, SaaS/Cloud Product, Information Technology & Services
Paytm
Impetus
Paytm, Paytm, Impetus, Cognizant, Accenture, Accenture, Accenture, Tolexo Online Pvt. Ltd.
Job Title : Senior Technical Data Engineering Lead
Company name : Paytm
Period : April 2023 - Present
Job Title : Technical Data Engineering Lead
Company name : Paytm
Period : January 2022 - April 2023
Summary : As a Lead of core merchant data team, I have been involved in planning tasks prioritization, optimization and improvement in current processes (both functionally and technically), designing, providing solutions to different teams (both High level and low level), leading and solving team problems and sprint goals, cross collaborating with teams (product and external) to understand and get those worked out along with further planning of tasks and their implementation for the quarter which would provide growth for the business as a whole.
Job Title : Senior Software Engineer
Company name : Impetus
Period : February 2020 - January 2022
Summary : Client: US Based airline client
Aim: The capabilities built were in silos and needed to be integrated on AWS cloud from on-prem.
Created a template parser framework using Python 3.7 to validate the metadata and also the data which is received, processed and transformed using Apache Spark 2.4.5 from client’s end.The processed file is then kept in curated bucket from raw bucket once the AWS glue job is completed successfully.
The data generated is then crawled and queried using AWS Athena.
The validation of initial data is done using JSON generated via Lambda function.
Managed a team and worked on gathering client requirements, maintaining agile based sprint using JIRA and other project management related activities.
Client: Intelligence and Investigation domain client
Aim: A quicker search for the most queried parameters was needed and therefore an effective search engine solution needed to be put into place.
The sensitive data was consumed from Kafka and all the searchable components were queried using Apache Solr and ELK stack. All the less searchable parameters were stored as a part of a data store in Apache Cassandra and benchmarking on various parameters were done.
The scale of data needed to be produced by creating an extrapolation utility which could create millions of records in less than a minute in both JSON and CSV format.
Internal:
Created a sentiment analyser to gauge the sentiments of internal applications using NLP and NLTK. The performance in terms of accuracy turned out to be better than Vader sentiment analyser.
Did a couple of other machine learning POCs and initiatives.
Location : Gurgaon, India
Job Title : Associate
Company name : Cognizant
Period : April 2018 - January 2020
Summary : Client: Financial service client-European Bank
Project: AML (Anti Money Laundering)
Apr 2018 to Present
Aim: To reduce the false positives of the model from 91% and increasing the reporting likelihood to the investigators from
9% by reducing a monthly backlog and reducing the SLA of 30 days for investigation.
Worked in ingestion of data from various sources such as Oracle, Teradata, sandpits, etc. using Sqoop, coordinated by
Oozie, processed them using Hive/Impala, pre-processed using Spark and Paylay model was trained using LightGBM
algorithm to send a final model csv file to stakeholders.
Worked on Hue and CDH distribution and autosys was used for end to end scheduling of jobs
Project: ODS (Operational Data Store)
Aim: The aim is to create a single data store for all mainframe data such that a single version of truth is presented and
consumed by tenants.
Custdb data of mainframes is consumed by Kafka using CDC-Attunity, transformed using pyspark, java using maven
build tool and dumped into MongoDB as a part of Operational Data Store.
Location : Gurgaon, Haryana, India
Job Title : Software Engineering Analyst
Company name : Accenture
Period : November 2016 - March 2018
Summary : • Worked in the ingestion of telecom data in the form of CDRs and UDRs belonging from different domains and regions in data collection layer using Apache Kafka.
• Data was then transformed using Apache Spark which served as the processing layer
• Apache Hive was being used as the storage layer of data records.
• Sound analytical and troubleshooting skills
Location : Gurgaon, India
Job Title : Software Engineering Analyst
Company name : Accenture
Period : May 2016 - October 2016
Summary : • I have close to 3 years of experience in CMT (Communication, media and technology) industry with proficiency of testing in transformation projects. I was involved in working with diverse modules of transformation portfolios and lead the MVNO.
• Worked as an automation test engineer in Selenium Webdriver using Maven build and dependencies, CI-Jenkins, Version control- GIT, pom based TestNG framework with a data driven approach. Also worked for mobile automation using Appium.
Location : Pune Area, India
Job Title : Associate Software Engineer
Company name : Accenture
Period : October 2014 - April 2016
Summary : • Worked in a sprint team for sprint testing, application tests, unit and system tests while working closely with Scrum stakeholders
• Worked in Agile, Waterfall, Scrums and well versed with SDLC lifecycle.
• Well versed and experienced with best practices, processes such as lean to optimize and make the system smooth and transparent
• Initiatives: EggPlant functional, ghostLab.
• Tools: HP ALM, Rally,JIRA, NVDA, Eclipse, Visual Studio Code, Mercury Loadrunner, SoapUI, Postman, Tosca
Location : Gurgaon, India
Job Title : Associate Software Programmer
Company name : Tolexo Online Pvt. Ltd.
Period : April 2014 - September 2014
Summary : As a fresher worked in B2B e-commerce startup - Tolexo (a subsidiary of Indiamart Intermesh Pvt. Ltd.)
Location : Noida, Uttar Pradesh, India
Title : Verified International Academic Qualifications
Period : July 2021 - Present
Summary : credly.com, https://www.credly.com/badges/fc1d5e52-8906-4d0e-874d-ab6e8d0bab6e?source=linked_in_profile
Issuing Authority : World Education Services
Title : Confluent Fundamentals for Apache Kafka
Period : February 2021 - Present
Summary : ee955a0f-0a56-463d-b167-3fd8726c7cb4?_ga=2.113805058., credential.net, https://www.credential.net/ee955a0f-0a56-463d-b167-3fd8726c7cb4?_ga=2.113805058.406690295.1612932891-2077559474.1612932891#gs.sjlwoz
Issuing Authority : Confluent
Title : Microsoft Certified: Azure Fundamentals
Period : August 2020 - Present
Summary : c84321dc-aa4e-465c-b495-49604d8b7fd1, youracclaim.com, https://www.youracclaim.com/badges/c84321dc-aa4e-465c-b495-49604d8b7fd1
Issuing Authority : Microsoft
Title : Databricks Spark SQL
Period : March 2020 - Present
Summary : databricks.com, https://academy.databricks.com/award/completion/d228b9ca-ba6b-30e7-beeb-264964c5497e
Issuing Authority : Databricks
Title : Databricks Spark Structured Streaming
Period : March 2020 - Present
Summary : databricks.com, https://academy.databricks.com/award/completion/a6c347cb-75fd-3969-a772-c5b4a7a7b32d
Issuing Authority : Databricks
Title : Spark - Level 1
Period : October 2019 - Present
Summary : youracclaim.com, https://www.youracclaim.com/badges/77028e40-e21d-4596-9820-c2ffa141c223/linked_in_profile
Issuing Authority : IBM
Title : Vantage Certified Associate
Period : June 2020 - June 2025
Summary : youracclaim.com, https://www.youracclaim.com/badges/17b44676-6266-4036-ae7c-b3be984e2a2c/linked_in_profile
Issuing Authority : Teradata
Title : Databricks Certified Associate Developer for Apache Spark 3.0
Period : July 2022 - July 2024
Summary : 55277330, databricks.com, https://credentials.databricks.com/043d404f-d29b-42d0-bcc1-3a1c972045c0
Issuing Authority : Databricks
Title : DBT Fundamentals
Period : May 2022 - May 2024
Summary : f39e503c-a02b-4761-97b1-eec41162e9eb, credential.net, https://www.credential.net/f39e503c-a02b-4761-97b1-eec41162e9eb
Issuing Authority : dbt Labs
Title : Databricks Lakehouse Fundamentals
Period : August 2022 - August 2023
Summary : 955c3b29-b46f-43f3-8b26-04db171d5c37, databricks.com, https://credentials.databricks.com/955c3b29-b46f-43f3-8b26-04db171d5c37
Issuing Authority : Databricks
Title : Sun Certified Java Developer (SCJD)
Issuing Authority : Sun Microsystems
Organization name : Techinnovative Position : Freelance Consultant Period : January 2017 - Present Organization name : UpGrad Position : Coach Period : May 2021 - October 2021 Organization name : EduCBA Position : Freelance Technical Content Writer
Show More