Data Engineer (Python)

This job is no longer open

Who is Softway?

Softway is a business-to-employee solutions company.We work with businesses to impact the everyday behaviors, mindsets, and attitudes and help drive valuable changes within.Faster innovation and more inclusion to lead cultural and digital transformations and make human-focused technology to propel businesses. Our solutions include culture & inclusion services and products, technology experiences, and communications. Ultimately, we believe love is a business strategy. Learn more: Visit us: https://www.softway.com/Our Podcast : https://www.softway.com/laabsBuy our book: https://www.loveasabusinessstrategy.com/Who are we looking for ?Softway is on the lookout for a Data Engineer in Python who is passionate about building products that our customers love.

You will join a dynamic and fast-paced environment and work with cross-functional teams to design, build and roll-out products that deliver the company’s vision and strategy.We're seeking a Python-savvy Data Engineer with a growing interest in big data technologies to join our dynamic team. You'll use Python to build essential data pipelines, wrangle datasets, and collaborate with our data science team. This is a fantastic opportunity to gain experience with PySpark and other in-demand tools while contributing to our data-driven initiatives.Responsibilities- Design and develop Python-based ETL/ELT processes, incorporating PySpark for larger datasets.- Clean, transform, and prepare data for analysis using Python (Pandas, NumPy) and PySpark.- Work with structured, semi-structured, and unstructured data from various sources- Write SQL queries to effectively interact with relational databases.- Collaborate effectively with team membersfrom different disciplines, such as data science and business intelligence, to design and implement data solutions based on provided specifications.- Explore and utilize cloud services for data storage and processing on AWS- Learn and contribute to best practices in data pipeline development and maintenance considering optimizing performance and ensuring scalability.- Lead development of entire data systems/subsystems and implement data pipelines- Contribute to data system design and architecture along with working closely with various engineers to ensure that designs align with overall data architecture and scalability requirements- Engage in client requirement gathering, solution presentation and feedback gathering to revise the solutions.- Mentor team members and share knowledge fostering a culture of mentorship within the team, where knowledge and expertise are freely exchanged.Requirements- Strong Python programming fundamentals and knowledge of core data manipulation libraries (Pandas, NumPy).- Basic understanding of PySpark for distributed data processing.- Proficiency in SQL for querying and working with relational data.- Familiarity with data warehousing and ETL/ELT principles.- Willingness to learn and grow within a collaborative environment.- Collaborate effectively with cross-functional teams to translate business requirements into solutions. - Communicate technical concepts to diverse audiences- Stay updated on industry trends and best practices- Propose innovative solutions for business/technical challenges - Demonstrate expertise and contribute to technical discussions- Set up coding standards, best practices, and the overall architecture of the system for data solutions- Drive innovation within the data engineering team and the organization.

Identify areas for improvement and propose innovative solutions to address business needs or technical challenges. - Should be leading by example in experimenting with new technologies and methodologies, evaluating their feasibility and potential impact

Great to Have

Passion to stay up-to-date with technology and trendsExpertise in PySpark.Knowledge on different cloud services for data storage, processing and analytics from multiple cloud providers.

Why should you work at Softway?

Softway’s cross-functional team structure, varied domains and flat hierarchy offer much exposure and learning from a bird's eye-view. You’d love the interaction with our talented folks. Softway gives you ample learning opportunities, pushing you while having your back. We work hard and party harder !! We encourage you to voice your opinions and yes, we actually listen. We take pride in having an ego-less workforce, it allows us to focus on what we’re good at - bringing solutions to life. In addition to a competitive salary and great work culture, we offer GREAT FLEXIBILITY with Flexible Time Offs.

Submit Your Application

You have successfully applied

You have errors in applying

Apply With Resume *First Name*Middle NameLast Name*Email*MobilePhoneReferral *--Choose--YESNOReferred by: Name & comments if any: Social Network and Web LinksProvide us with links to see some of your work (Git/ Dribble/ Behance/ Pinterest/ Blog/ Medium) + Employer + Add Employer Education + Add Education How soon can you join us ( Notice Period)*--Choose--Immediate < 15 days30 days45 days 60 days 90 days and aboveCurrent CTC*Expected CTC*How many years of experience do you have in Data Engineering ?*--Choose--2 years3 years 4 years 5 years & Above How many years of experience do you have in Python?*--Choose--2 years3 years 4 years 5 years & Above How many years of experience do you have handling data pipelines?*--Choose--2 years3 years4 years & above How many years experience in handling Big Data?*--Choose--1 year2 years3 years & above Do you have experience in PySpark ?*--Choose--YesNoIf not already in Bangalore, would you be willing to relocate? *Base (India)*--Choose--North East West SouthWhat inspired you to apply for this role at Softway? *

{ "@context" : "http://schema.org/", "@type" : "JobPosting", "url" : "https://softwaysolutions.freshteam.com/jobs/zhhMTIFE0wws/Data%20Engineer%20(Python)", "title" : "Data Engineer (Python)", "description" : "<h1 dir="ltr" style="line-height:2.16;margin-top:20pt;margin-bottom:6pt;"><span style="font-size:10pt;font-family:Arial,sans-serif;color:#000000;background-color:transparent;font-weight:700;font-style:normal;font-variant:normal;text-decoration:none;vertical-align:baseline;white-space:pre;white-space:pre-wrap;">Who is Softway?</h1><span style="font-size:10pt;font-family:Arial,sans-serif;color:#000000;background-color:transparent;font-weight:400;font-style:normal;font-variant:normal;text-decoration:none;vertical-align:baseline;white-space:pre;white-space:pre-wrap;">Softway is a business-to-employee solutions company.We work with businesses to impact the everyday behaviors, mindsets, and attitudes  and help drive valuable changes within.Faster innovation and more inclusion to lead cultural and  digital transformations and make human-focused technology to propel businesses.

Our solutions include culture & inclusion services and products, technology experiences, and communications. Ultimately, we believe love is a business strategy. <span style="font-size:10pt;font-family:Arial,sans-serif;color:#000000;background-color:transparent;font-weight:400;font-style:normal;font-variant:normal;text-decoration:none;vertical-align:baseline;white-space:pre;white-space:pre-wrap;"> <span style="font-size:10pt;font-family:Arial,sans-serif;color:#000000;background-color:transparent;font-weight:700;font-style:normal;font-variant:normal;text-decoration:none;vertical-align:baseline;white-space:pre;white-space:pre-wrap;">Learn more: <span style="font-size:10pt;font-family:Arial,sans-serif;color:#000000;background-color:transparent;font-weight:400;font-style:normal;font-variant:normal;text-decoration:none;vertical-align:baseline;white-space:pre;white-space:pre-wrap;">Visit us: https://www.softway.com/<span style="font-size:10pt;font-family:Arial,sans-serif;color:#000000;background-color:transparent;font-weight:400;font-style:normal;font-variant:normal;text-decoration:none;vertical-align:baseline;white-space:pre;white-space:pre-wrap;">Our Podcast : https://www.softway.com/laabs<span style="font-size:10pt;font-family:Arial,sans-serif;color:#000000;background-color:transparent;font-weight:400;font-style:normal;font-variant:normal;text-decoration:none;vertical-align:baseline;white-space:pre;white-space:pre-wrap;">Buy our book:  https://www.loveasabusinessstrategy.com/<span style="font-size:10pt;font-family:Arial,sans-serif;color:#000000;background-color:transparent;font-weight:400;font-style:normal;font-variant:normal;text-decoration:none;vertical-align:baseline;white-space:pre;white-space:pre-wrap;"> <span style="font-size:10pt;font-family:Arial,sans-serif;color:#000000;background-color:transparent;font-weight:700;font-style:normal;font-variant:normal;text-decoration:none;vertical-align:baseline;white-space:pre;white-space:pre-wrap;">Who are we looking for ?<span style="font-size:10pt;font-family:Arial,sans-serif;color:#000000;background-color:transparent;font-weight:400;font-style:normal;font-variant:normal;text-decoration:none;vertical-align:baseline;white-space:pre;white-space:pre-wrap;">Softway is on the lookout for a  Data Engineer in Python who is passionate about building products that our customers love.

You will join a dynamic and fast-paced environment and work with cross-functional teams to design, build and roll-out products that deliver the company’s vision and strategy. <span style="font-size:10pt;font-family:Arial,sans-serif;color:#000000;background-color:transparent;font-weight:400;font-style:normal;font-variant:normal;text-decoration:none;vertical-align:baseline;white-space:pre;white-space:pre-wrap;">We're seeking a Python-savvy Data Engineer with a growing interest in big data technologies to join our dynamic team.

You'll use Python to build essential data pipelines, wrangle datasets, and collaborate with our data science team. This is a fantastic opportunity to gain experience with PySpark and other in-demand tools while contributing to our data-driven initiatives.<div> </div><span style="font-size:10pt;font-family:Arial,sans-serif;color:#000000;background-color:transparent;font-weight:700;font-style:normal;font-variant:normal;text-decoration:none;vertical-align:baseline;white-space:pre;white-space:pre-wrap;">Responsibilities<span style="font-size:10pt;font-family:Arial,sans-serif;color:#000000;background-color:#ffffff;font-weight:400;font-style:normal;font-variant:normal;text-decoration:none;vertical-align:baseline;white-space:pre;white-space:pre-wrap;">- Design and develop Python-based ETL/ELT processes, incorporating PySpark for larger datasets.<span style="font-size:10pt;font-family:Arial,sans-serif;color:#000000;background-color:#ffffff;font-weight:400;font-style:normal;font-variant:normal;text-decoration:none;vertical-align:baseline;white-space:pre;white-space:pre-wrap;">- Clean, transform, and prepare data for analysis using Python (Pandas, NumPy) and PySpark.<span style="font-size:10pt;font-family:Arial,sans-serif;color:#000000;background-color:#ffffff;font-weight:400;font-style:normal;font-variant:normal;text-decoration:none;vertical-align:baseline;white-space:pre;white-space:pre-wrap;">- Work with structured, semi-structured, and unstructured data from various sources<span style="font-size:10pt;font-family:Arial,sans-serif;color:#000000;background-color:#ffffff;font-weight:400;font-style:normal;font-variant:normal;text-decoration:none;vertical-align:baseline;white-space:pre;white-space:pre-wrap;">- Write SQL queries to effectively interact with relational databases.<span style="font-size:10pt;font-family:Arial,sans-serif;color:#000000;background-color:#ffffff;font-weight:400;font-style:normal;font-variant:normal;text-decoration:none;vertical-align:baseline;white-space:pre;white-space:pre-wrap;">- Collaborate <span style="font-size:10pt;font-family:Arial,sans-serif;color:#000000;background-color:transparent;font-weight:400;font-style:normal;font-variant:normal;text-decoration:none;vertical-align:baseline;white-space:pre;white-space:pre-wrap;">effectively with team members<span style="font-size:10pt;font-family:Arial,sans-serif;color:#38761d;background-color:transparent;font-weight:400;font-style:normal;font-variant:normal;text-decoration:none;vertical-align:baseline;white-space:pre;white-space:pre-wrap;"> <span style="font-size:10pt;font-family:Arial,sans-serif;color:#000000;background-color:transparent;font-weight:400;font-style:normal;font-variant:normal;text-decoration:none;vertical-align:baseline;white-space:pre;white-space:pre-wrap;">from different disciplines, such as data science and business intelligence,<span style="font-size:10pt;font-family:Arial,sans-serif;color:#000000;background-color:#ffffff;font-weight:400;font-style:normal;font-variant:normal;text-decoration:none;vertical-align:baseline;white-space:pre;white-space:pre-wrap;"> to design and implement data solutions based on provided specifications.<span style="font-size:10pt;font-family:Arial,sans-serif;color:#000000;background-color:#ffffff;font-weight:400;font-style:normal;font-variant:normal;text-decoration:none;vertical-align:baseline;white-space:pre;white-space:pre-wrap;">- Explore and utilize cloud services for data storage and processing on AWS<span style="font-size:10pt;font-family:Arial,sans-serif;color:#000000;background-color:#ffffff;font-weight:400;font-style:normal;font-variant:normal;text-decoration:none;vertical-align:baseline;white-space:pre;white-space:pre-wrap;">- Learn and contribute to best practices in data pipeline development and maintenance considering <span style="font-size:10pt;font-family:Arial,sans-serif;color:#000000;background-color:transparent;font-weight:400;font-style:normal;font-variant:normal;text-decoration:none;vertical-align:baseline;white-space:pre;white-space:pre-wrap;">optimizing performance and ensuring scalability<span style="font-size:10pt;font-family:Arial,sans-serif;color:#000000;background-color:#ffffff;font-weight:400;font-style:normal;font-variant:normal;text-decoration:none;vertical-align:baseline;white-space:pre;white-space:pre-wrap;">.<span style="font-size:10pt;font-family:Arial,sans-serif;color:#000000;background-color:#ffffff;font-weight:400;font-style:normal;font-variant:normal;text-decoration:none;vertical-align:baseline;white-space:pre;white-space:pre-wrap;">- Lead development of entire data systems/subsystems and implement data pipelines<span style="font-size:10pt;font-family:Arial,sans-serif;color:#000000;background-color:#ffffff;font-weight:400;font-style:normal;font-variant:normal;text-decoration:none;vertical-align:baseline;white-space:pre;white-space:pre-wrap;">- Contribute to data system design and architecture along with <span style="font-size:10pt;font-family:Arial,sans-serif;color:#000000;background-color:transparent;font-weight:400;font-style:normal;font-variant:normal;text-decoration:none;vertical-align:baseline;white-space:pre;white-space:pre-wrap;">working closely with various engineers to ensure that designs align with overall data architecture and scalability requirements<span style="font-size:10pt;font-family:Arial,sans-serif;color:#000000;background-color:#ffffff;font-weight:400;font-style:normal;font-variant:normal;text-decoration:none;vertical-align:baseline;white-space:pre;white-space:pre-wrap;">- Engage in client requirement gathering, solution presentation and feedback gathering to revise the solutions.<span style="font-size:10pt;font-family:Arial,sans-serif;color:#000000;background-color:#ffffff;font-weight:400;font-style:normal;font-variant:normal;text-decoration:none;vertical-align:baseline;white-space:pre;white-space:pre-wrap;">- Mentor team members and share knowledge <span style="font-size:10pt;font-family:Arial,sans-serif;color:#000000;background-color:transparent;font-weight:400;font-style:normal;font-variant:normal;text-decoration:none;vertical-align:baseline;white-space:pre;white-space:pre-wrap;">fostering a culture of mentorship within the team, where knowledge and expertise are freely exchanged.<div> </div><span style="font-size:10pt;font-family:Arial,sans-serif;color:#000000;background-color:#ffffff;font-weight:700;font-style:normal;font-variant:normal;text-decoration:none;vertical-align:baseline;white-space:pre;white-space:pre-wrap;">Requirements<div> </div><div><span style="font-size:10pt;font-family:Arial,sans-serif;color:#000000;background-color:#ffffff;font-weight:400;font-style:normal;font-variant:normal;text-decoration:none;vertical-align:baseline;white-space:pre;white-space:pre-wrap;">- Strong Python programming fundamentals and knowledge of core data manipulation libraries (Pandas, NumPy).</div><span style="font-size:10pt;font-family:Arial,sans-serif;color:#000000;background-color:#ffffff;font-weight:400;font-style:normal;font-variant:normal;text-decoration:none;vertical-align:baseline;white-space:pre;white-space:pre-wrap;">- Basic understanding of PySpark for distributed data processing.<span style="font-size:10pt;font-family:Arial,sans-serif;color:#000000;background-color:#ffffff;font-weight:400;font-style:normal;font-variant:normal;text-decoration:none;vertical-align:baseline;white-space:pre;white-space:pre-wrap;">- Proficiency in SQL for querying and working with relational data.<span style="font-size:10pt;font-family:Arial,sans-serif;color:#000000;background-color:#ffffff;font-weight:400;font-style:normal;font-variant:normal;text-decoration:none;vertical-align:baseline;white-space:pre;white-space:pre-wrap;">- Familiarity with data warehousing and ETL/ELT principles.<span style="font-size:10pt;font-family:Arial,sans-serif;color:#000000;background-color:#ffffff;font-weight:400;font-style:normal;font-variant:normal;text-decoration:none;vertical-align:baseline;white-space:pre;white-space:pre-wrap;">- Willingness to learn and grow within a collaborative environment.<span style="font-size:10pt;font-family:Arial,sans-serif;color:#000000;background-color:#ffffff;font-weight:400;font-style:normal;font-variant:normal;text-decoration:none;vertical-align:baseline;white-space:pre;white-space:pre-wrap;">- Collaborate effectively with cross-functional teams <span style="font-size:10pt;font-family:Arial,sans-serif;color:#000000;background-color:transparent;font-weight:400;font-style:normal;font-variant:normal;text-decoration:none;vertical-align:baseline;white-space:pre;white-space:pre-wrap;">to translate business requirements into solutions. <span style="font-size:10pt;font-family:Arial,sans-serif;color:#000000;background-color:#ffffff;font-weight:400;font-style:normal;font-variant:normal;text-decoration:none;vertical-align:baseline;white-space:pre;white-space:pre-wrap;">- Communicate technical concepts to diverse audiences<span style="font-size:10pt;font-family:Arial,sans-serif;color:#000000;background-color:#ffffff;font-weight:400;font-style:normal;font-variant:normal;text-decoration:none;vertical-align:baseline;white-space:pre;white-space:pre-wrap;">- Stay updated on industry trends and best practices<span style="font-size:10pt;font-family:Arial,sans-serif;color:#000000;background-color:#ffffff;font-weight:400;font-style:normal;font-variant:normal;text-decoration:none;vertical-align:baseline;white-space:pre;white-space:pre-wrap;">- Propose innovative solutions for business/technical challenges<span style="font-size:10pt;font-family:Arial,sans-serif;color:#000000;background-color:#ffffff;font-weight:400;font-style:normal;font-variant:normal;text-decoration:none;vertical-align:baseline;white-space:pre;white-space:pre-wrap;"> - Demonstrate expertise and contribute to technical discussions<span style="font-size:10pt;font-family:Arial,sans-serif;color:#000000;background-color:transparent;font-weight:400;font-style:normal;font-variant:normal;text-decoration:none;vertical-align:baseline;white-space:pre;white-space:pre-wrap;">- Set up coding standards, best practices, and the overall architecture of the system for data solutions<span style="font-size:10pt;font-family:Arial,sans-serif;color:#000000;background-color:transparent;font-weight:400;font-style:normal;font-variant:normal;text-decoration:none;vertical-align:baseline;white-space:pre;white-space:pre-wrap;">- Drive innovation within the data engineering team and the organization.

Identify areas for improvement and propose innovative solutions to address business needs or technical challenges. <span style="font-size:10pt;font-family:Arial,sans-serif;color:#000000;background-color:transparent;font-weight:400;font-style:normal;font-variant:normal;text-decoration:none;vertical-align:baseline;white-space:pre;white-space:pre-wrap;">- Should be leading by example in experimenting with new technologies and methodologies, evaluating their feasibility and potential impact Great to Have Passion to stay up-to-date with technology and trends Expertise in PySpark. Knowledge on different cloud services for data storage, processing and analytics from multiple cloud providers. Why should you work at Softway? Softway’s cross-functional team structure, varied domains and flat hierarchy offer much exposure and learning from a bird's eye-view.

You’d love the interaction with our talented folks. Softway gives you ample learning opportunities, pushing you while having your back. We work hard and party harder !! We encourage you to voice your opinions and yes, we actually listen. We take pride in having an ego-less workforce, it allows us to focus on what we’re good at - bringing solutions to life. In addition to a competitive  salary and great work culture, we offer GREAT FLEXIBILITY with Flexible Time Offs. <div> </div>", "datePosted" : "2024-08-23 06:51:24 UTC", "employmentType" : "FULL_TIME", "remote" : "false", "hiringOrganization" : { "@type":"Organization", "name":"Softway" }, "jobLocation" : { "@type": "Place", "address": { "@type": "PostalAddress", "streetAddress": "", "addressRegion": "Bengaluru", "postalCode": "", "addressLocality": "", "addressCountry": "India" } } } var translation = { 'employer_title': `Designation`, 'employer_company': `Company/Business name`, 'employer_start_date': `Start date`, 'employer_end_date': `End date`, 'employer_is_current': `I currently work here`, 'employer_summary': `Summary`, 'employer_remove': `Remove this employer`, 'education_degree': `Degree`, 'education_field_of_study': `Field of study/major`, 'education_school_name': `Institution/school name`, 'education_start_date': `Start date`, 'education_end_date': `End date`, 'education_is_current': `Currently pursuing`, 'education_grade': `Grade`, 'education_remove': `Remove this degree` };

This job is no longer open

Your tracker settings

We use cookies and similar methods to recognize visitors and remember their preferences. We also use them to measure ad campaign effectiveness, target ads and analyze site traffic. To learn more about these methods, including how to disable them, view our Cookie Policy or Privacy Policy.

By tapping `Accept`, you consent to the use of these methods by us and third parties. You can always change your tracker preferences by visiting our Cookie Policy.

ThatStartupJob

Discover the best startup and their job positions, all in one place.