For Kenny Shin, the chief technology officer of direct-to-consumer alternative asset management company Fundrise, some of the most exciting technological advancements in recent years have been in structured query language tools. Despite its decades-old roots, SQL is undergoing a renaissance, buoyed by SQL tool advancements in a variety of data-related tools.
Image by Freepik
“If you dig deeper into what’s going on right now, SQL — combined with cloud computing, serverless databases, and data warehouses — is yielding a new set of tools that take advantage of the ubiquity of SQL but have abstractions that make it possible to utilize SQL at a complexity far beyond how an individual would responsibly query a data source to produce wonderfully seamless solutions,” Shin explained in a recent interview with Tech Brew.
What Is SQL?
SQL is a standardized programming language specifically designed for managing and interacting with relational databases, where information is stored in interrelated tables. Crucially, SQL operates through declarative statements, allowing users to specify what data they want to retrieve or manipulate without detailing the procedural steps for achieving these tasks. It’s essential in relational database management, enabling operations like querying for data subsets, updating records, and managing database structures.
SQL’s strength lies in its ability to handle complex queries over large datasets. It excels in aggregating, sorting, and filtering data efficiently, crucial for applications in data analysis, business intelligence, and backend systems of many web services. Its syntax, while straightforward for basic operations, can be optimized for performance and complex data relationships, making it an indispensable tool for database administrators and developers dealing with data-driven applications.
Recent Advancements
Historically viewed as a straightforward query language, SQL is now at the heart of more complex operations, thanks to its integration with newer technologies.
Shin mentioned Dbt, Fivetran, Snowflake, and Looker as some of the new data management technologies implemented at Fundrise.
Dbt makes it easier for users to transform raw data using SQL, and Fivetran automatically moves data from various sources into a data warehouse, simplifying the preparation process for analysis. Snowflake is a cloud-based platform that enhances SQL’s data storage and processing, allowing for easy scaling and efficient data management, and Looker extends SQL to provide advanced data visualization and reporting.
“The productivity that these tools unlock for us is remarkable, especially to people used to more heavily homegrown data engineering environments,” said Shin.
Ultimately, these tools enable organizations to process large datasets more quickly and with greater flexibility. They also make it easier to integrate data from various sources, a critical capability in an era where data is often scattered across multiple systems and platforms.
Furthermore, the integration of SQL with cloud computing has opened up new possibilities for data storage and access. Serverless databases and cloud-based data warehouses offer scalability and cost-effectiveness, making them ideal for businesses that need to manage large volumes of diverse data without the burden of maintaining physical infrastructure.
Fundrise and Dbt: A Case Study
Fundrise has data needs ranging from financial data about its investments, assets, and transactions, to data on the effectiveness of its marketing campaigns.
In a recent article, Charles Wood, VP of analytics, and Jack Ploshnick, analytics manager at Fundrise, explained how the transition to Dbt has helped them manage this complex array of data.
“Dbt allows us to create new business intelligence dashboards that never existed before,” said Ploshnick. “More importantly, it allows us to spend much less time creating those dashboards and more time on in-depth analysis. We have more time to answer the harder, impactful questions.”
Wood added, “There was a huge opportunity cost associated with losing out on the highest-value work that we could be doing just to keep the system going. The amount of time we spend on maintenance is almost zero now.”
They went on to explain that Dbt enabled the Fundrise marketing team to more efficiently and accurately analyze the impact of their advertisements in terms of costs versus revenue added.
“In the past, we had to check in every day to make sure the marketing attribution numbers were correct and that all the data was flowing through as it was supposed to,” said Ploshnick. “If the data source says that a particular campaign was efficient, we would need to go in and spend time making sure that number was actually right.
“That is now zero work. We know the dashboard is correct. The question now becomes: Why is the marketing program efficient or why is it inefficient? We can finally focus on those kinds of questions.”
The transformation driven by advanced SQL tools at Fundrise is indicative of a broader shift in the way companies are managing data. Tools like Dbt are transforming labor-intensive data management into more efficient, automated, and insightful processes. Businesses across industries will only improve in their ability to leverage these tools as the technology improves, harnessing the full potential of SQL to turn complex data challenges into opportunities for innovation and growth.
