Dattell report reveals most popular technologies for data engineers – TechRepublic
Register for your free TechRepublic membership or if you are already a member, sign in using your preferred method below.
We recently updated our Terms and Conditions for TechRepublic Premium. By clicking continue, you agree to these updated terms.
Invalid email/username and password combination supplied.
An email has been sent to you with instructions on how to reset your password.
By registering, you agree to the Terms of Use and acknowledge the data practices outlined in the Privacy Policy.
You will also receive a complimentary subscription to TechRepublic’s News and Special Offers newsletter and the Top Story of the Day newsletter. You may unsubscribe from these newsletters at any time.
All fields are required. Username must be unique. Password must be a minimum of 6 characters and have any 3 of the 4 items: a number (0 through 9), a special character (such as !, $, #, %), an uppercase character (A through Z) or a lowercase (a through z) character (no spaces).
Dattell report reveals most popular technologies for data engineers
Your email has been sent
Discover some of the most popular tools and technologies desired by employers looking to fill data engineering jobs, according to Dattell.
Demand for technology professionals has been rising after a brief downturn following the start of the COVID-19 pandemic. And one tech job with slots to fill throughout the industry is that of data engineer. Responsible for creating the infrastructure needed to consume, process and store vast amounts of information, data engineers are seeing an increase not just in job openings but in the tools that help them do their work.
A report released June 29, 2022, by data engineering provider Dattell looks at the current market for this career. To compile its findings, Dattell collected and analyzed a huge amount of data itself, matching specific technologies with the number of job openings.
Eyeing the job market for data engineers, Dattell analyzed 340,000 different job postings. Among these, 35% were for positions in data orchestration, 30% for data storage, 29% for data visualization, and 6% for data processing.
Additionally, Dattell analyzed the languages used by data engineers. Python was the most popular, preferred by employers 38% of the time, followed by Java at 33% and SQL at 22%. Combined, the three of them account for 550,000 job openings.
SEE: Hiring Kit: Database engineer (TechRepublic Premium)
Moreover, compensation for data engineering jobs varied from $60,000 as a starting salary to as high as $180,000 for more advanced positions. The highest compensations were offered to those working with Kubernetes, Elasticsearch, PostgreSQL and Terraform, with salaries landing at $140,000 and higher. And while there are a healthy amount of openings for those working with Tableau and Power BI, most of them offer less than $100,000.
In its research, Dattell compiled a list of the top 20 most popular data engineering technologies, pointing to such tools as Microsoft Power BI, Terraform, Chef, Spark, Elasticsearch, Hadoop and Kafka. The products cited offer support in areas spanning data storage, data orchestration, data processing and data visualization, indicating that no single segment of data engineering dominates the rest.
Based on Dattell’s research, the five most popular data engineering tools are MongoDB, Tableau, Kubernetes, PostgreSQL and Ansible, covering data storage, data orchestration and data visualization technologies.
Among the tools themselves, Tableau and Kubernetes took the top spots for the greatest number of job openings. Other tools generating a high number of open jobs were Ansible, Hadoop, Terraform, Splunk, Power BI, MongoDB and PostgreSQL.
And among the data orchestration tools examined, Kubernetes is by far the leader, followed by Ansible. Though both products fall into the same space, each one is used differently. Kubernetes lets professionals manage and maintain container health, while Ansible allows them to deploy configuration changes and manage updates.
With so many free products available, employers don’t see a great need to pay for data processing tools, according to Dattell. As a result, companies are looking for employees and consultants with expertise in free and open-source technologies, such as leading tools like Apache Spark and Apache Kafka.
For example, with data storage technologies, paid tools are preferred by 59% of employers, leaving a still hefty 39% who prefer free tools. Among the most popular data storage technologies analyzed by Dattell, MongoDB was the most popular, followed by PostgreSQL, both of which are free. Some products, such as Elasticsearch, come in both paid and free versions.
Learn the latest news and best practices about data science, big data analytics, and artificial intelligence.
Dattell report reveals most popular technologies for data engineers
Your email has been sent
Your message has been sent
TechRepublic Premium content helps you solve your toughest IT issues and jump-start your career or next project.
These 11 cloud-to-cloud solutions back up your organization’s data so you’ll be covered in the event of deletions, malware or outages. Compare the best online cloud backup services now.
You can use a mobile device to speak with another person directly through the Teams app. Lance Whitney shows you how to use this handy feature.
A phishing technique called Browser in the Browser (BITB) has emerged, and it’s already aiming at government entities, including Ukraine. Find out how to protect against this new threat.
With so many project management software options to choose from, it can seem daunting to find the right one for your projects or company. We’ve narrowed them down to these nine.
Start-ups, DARPA and Accenture Ventures announce research partnerships, new hardware and strategic investments.
IIoT software assists manufacturers and other industrial operations with configuring, managing and monitoring connected devices. A good IoT solution requires capabilities ranging from designing and delivering connected products to collecting and analyzing system data once in the field. Each IIoT use case has its own diverse set of requirements, but there are key capabilities and …
Recruiting an Operations Research Analyst with the right combination of technical expertise and experience will require a comprehensive screening process. This Hiring Kit provides an adjustable framework your business can use to find, recruit and ultimately hire the right person for the job.This hiring kit from TechRepublic Premium includes a job description, sample interview questions …
The digital transformation required by implementing the industrial Internet of Things (IIoT) is a radical change from business as usual. This quick glossary of 30 terms and concepts relating to IIoT will help you get a handle on what IIoT is and what it can do for your business.. From the glossary’s introduction: While the …
Procuring software packages for an organization is a complicated process that involves more than just technological knowledge. There are financial and support aspects to consider, proof of concepts to evaluate and vendor negotiations to handle. Navigating through the details of an RFP alone can be challenging, so use TechRepublic Premium’s Software Procurement Policy to establish …