Los Angeles County had an inefficient hiring process that often resulted in lag times of a year or more before positions were filled, so in 2021 its human resources department modernized its analytics system with an integration between Microsoft Azure and Databricks to improve hiring efficiency.
The Azure cloud platform includes such tools as Azure Data Lake, which customers can use to store and gain insights from large data sets, and Azure Synapse Analytics, a service that joins data integration, data warehousing and big data analytics.
Databricks' data lakehouse platform enables customers to query structured data with SQL as they would in a data warehouse as well as query unstructured data as they would within a data lake.
Starting in July 2021, with guidance from consulting firm Accenture, LA County -- the largest county in the U.S., with more than 10 million residents -- redesigned its analytics stack with the Azure Databricks Lakehouse in an attempt to improve its hiring practices.
Not enough time has passed to make year-over-year comparisons, but in less than a year the new technology stack has already resulted in not only a more streamlined hiring process but also cost savings, according to Majida Adnan, acting deputy general manager in information technology services for Los Angeles County.
"It's worked really well for us," she said recently during Data + AI Summit, a user conference hosted by Databricks.
Too much time
Before LA County overhauled the technology fueling its hiring process, the average time to hire was 384 days during the most recently completed fiscal year, according to Roozan Zarifian, chief information officer for the LA County human resources department.
LA County's HR department overhauled its analytics stack in order to improve the efficiency of its hiring process.
In fiscal 2019-2020 -- the last year before the COVID-19 pandemic -- LA County received more than 350,000 applications and filled more than 13,000 positions, with each hire taking an average of 327 days.
Over the next two fiscal years, though both the number of hires and applicants dropped due to the pandemic, the average time to hire jumped by more than 50 days.
"We work toward attracting and retaining a talented workforce," Zarifian said. "We want to be innovative, and we want to be flexible and transparent."
But before redesigning its HR analytics stack with Azure and Databricks, LA County's hiring process was inflexible. And a main reason was that its existing analytics technology was holding back the speed with which it could monitor its hiring practices, according to Zarifian.
Among the challenges the County struggled with were a cumbersome process for acquiring data, data managed in isolated repositories, difficulty cleansing and analyzing data, and difficulty sharing insights once data had been analyzed.
In addition, the County was dependent on its IT department for analysis, which led to long lag times between requests for information. And much of what needed to be done throughout the data management and analysis processes was manual, which also added time to examining processes and procedures.
Without the right tools, our resources spent the vast amount of their time cleaning data, identifying anomalies and then handling the exceptions. This semi-manual process of collecting, analyzing and reporting data was labor-intensive, inefficient, inconsistent, costly and prone to errors.
Roozan ZarifianChief information officer, LA County human resources department
Compounding LA County's problems are the complexities of hiring within such a large organization. The County requires exams for certain positions, has more than 1,200 job classifications and is subject to myriad civil service regulations related to hiring.
"Without the right tools, our resources spent the vast amount of their time cleaning data, identifying anomalies and then handling the exceptions," Zarifian said. "This semi-manual process of collecting, analyzing and reporting data was labor-intensive, inefficient, inconsistent, costly and prone to errors."
Objectives
In its effort to become more efficient, the HR department set out clear directives for what it wanted to accomplish -- the steps enabled by better technology that would cumulatively result in better efficiency -- that ultimately led to the integration between Azure and Databricks.
Most importantly, it wanted to identify the bottlenecks that led to such long lag times between the time a job came open and the time it was finally filled.
In addition, the department wanted to enhance the hiring experience of applicants, improve its access to talent so it could compete with other organizations for the best talent available, improve recruitment so it could proactively identify potential candidates, and increase the diversity of the talent its jobs attract.
"We want to gain insights about the candidates so we can continue to attract and retain a diverse and talented workforce," Zarifian said.
To meet those hiring goals, the department put in place a series of technology objectives, Zarifian continued.
The goal was to build an information loop that would feed on itself and improve with time and the addition of more data.
First, the County needed to automate the extract, transform and load process to ease the manual burden on data engineers. In addition, it needed to build real-time interactive dashboards that could update constantly with the most up-to-date data, automate anomaly detection, improve sharing and collaboration capabilities, and automatically capture data from multiple sources.
Five months after teaming up with Accenture to develop a new analytics stack, a set of interactive dashboards were ready for implementation in late 2021, and since then LA County has been in the process of transforming its hiring practices.
"We now have access to a wealth of information through these dashboards," Zarifian said. "We can see how long it takes to hire employees, and how long it takes to fill vacancies. We're also able to see how efficient we are in each step of the process. We know where the candidates are coming from, and which source yields the best candidates. We can also tell where in the process we are losing candidates."
The technology
Underlying the dashboards is an analytics stack featuring the Azure Databricks Lakehouse.
According to Adnan, LA County had four technological challenges it needed to address when developing its new analytics system and ultimately solving those challenges with a data lakehouse architecture.
It needed to develop a stack that was easy to use while also up to modern industry standards and capable of handling data sets of unlimited size. In addition, data acquisition had to be simple, requiring little manual ETL work, and it needed strong data governance capabilities and automated alerting capabilities.
Now, LA County's HR analytics stack begins with data ingestion into Azure Data Factory, which passes the raw data into Azure Data Lake. There, the Azure Databricks Lakehouse takes over as the data is prepared for analysis. Once the data is cleaned and integrated, it moves on to Azure Synapse and Power BI for analysis, insight and data-informed decision making.
"Data is the new oil, and as it is growing every day and multiplying, we wanted to make sure we designed a data system with a big data approach in mind," Adnan said.
And because it is designed for the cloud -- in addition to automating labor-intensive tasks that previously had to be done manually -- LA County's revamped HR analytics system is less expensive than it was before 2021, she continued.
"Moving these processes into the cloud, from an infrastructure perspective and from an automation perspective, there were a lot of costs that were saved, and the total cost of operations came down," Adnan said.
