Data science is a rapidly growing technology as organizations of all sizes embrace AI and ML, and with its growth comes no shortage of concerns.

The 2022 State of Data Science report, released today by data science platform vendor Anaconda, identifies key trends and concerns for data scientists and the organizations that employ them. Among the trends identified by Anaconda is the fact that the open source Python programming language continues to dominate the world of data science.

One of the main concerns identified in the report has to do with barriers to adoption of data science as a whole.

“One of the things that surprised me was that two-thirds of respondents said the biggest barrier to successful enterprise adoption of data science was the lack of data engineering and tools that enable them to generate better models. We felt that our investments were under-invested,” the founder told VentureBeat. “We’ve always known that data science and machine learning can suffer from poor models and inputs, but it’s great to see respondents rate this even higher than the talent-to-staff gap. It was interesting.”

AI bias is far from a solved problem

The problem of AI bias is a well-known problem in data science. Little is known about what organizations are actually doing to address this issue.

Last year, Anaconda’s 2021 State of Data Science found that 40% of organizations are planning or doing something to address the issue of bias. Rather than asking the same question this year, Anaconda decided to take a different approach.

“Rather than asking if the organization plans to address bias, we wanted to find out what specific steps the organization is currently taking to ensure fairness and reduce bias,” said Wang. said. “Last year’s findings showed that organizations had plans to address this, so we wanted to explore what actions organizations have taken and where their priorities lie for 2022. I did.”

As part of efforts to prevent AI bias, 31% of respondents say they evaluate data collection methods according to internally set standards of fairness. In contrast, 24% say they do not have fairness and debiasing criteria in their datasets and models.

AI explainability is a fundamental element that helps identify and prevent bias. When asked what tools are used for AI explainability, 35% of respondents said their organization runs a series of controlled tests to assess model explainability. 24% do not have the means or tools to ensure the explainability of their models.

“Although less than 50% of these efforts were implemented for each response measure, the results here show that organizations are taking a variety of approaches to mitigate bias,” says Wang. said Mr. “Finally, organizations are taking action. They are only at the beginning of their journey to address stigma.”

How data scientists spend their time

Data scientists have a variety of tasks that need to be performed as part of their job.

Actually deploying the model is the desired end goal, but that’s not where data scientists actually spend most of their time. In fact, the study found that data scientists spend only 9% of their time deploying models. Similarly, the respondent reports that he spends only 9% of his time on model selection.

The biggest time waster is data preparation and cleansing, which takes up 38% of the time.

A relationship of love and fear with open source

The report also asked how data scientists use and view open source software.

87% said their organization allows open source software. But despite its use, 54% of his respondents said they were concerned about open source security.

“Today, open source is embedded in almost every piece of software and technology, and not just because it’s cheaper in the long run,” Wang said. “All the innovation happening around AI, machine learning, and data science is happening within open source he ecosystems at a speed that closed systems can’t match.”

That said, Wang said it makes sense for organizations to recognize the risks associated with open source and develop plans to mitigate potential vulnerabilities.

“One of the advantages of open source is that patches and solutions are built in the open rather than behind closed doors,” he said.

Anaconda’s report is based on a survey of 3,493 respondents in 133 countries.