The 128th episode of Datacast is my conversation with Jason Risch, an investor on the enterprise team at Greylock - investing in security, AI/ML, data, infrastructure, and developer tools.
Our wide-ranging conversation touches on his upbringing in the Bay Area in the age of the “Moneyball” era, his undergraduate experience at Stanford, his early career at Opendoor/McKinsey/AI Fund, his decision to pursue venture capital at Greylock, his investments in enterprise data and AI products, Greylock’s support machine for founders, VC funding in the cloud, the next cloud data platform, and much more.
Please enjoy my conversation with Jason!
Timestamps
(01:45) Jason shared the formative experiences of his upbringing in the Bay Area and coming of age in the “Moneyball” era of baseball.
(05:03) Jason described his overall academic experience at Stanford - where he studied Mathematical and Computational Science with a minor in Classical Studies.
(09:15) Jason reflected on his experience participating in the Mayfield Fellowship at Stanford.
(12:03) Jason recalled his time being a part of the business operations team during a high-growth period at Opendoor.
(14:25) Jason talked about lessons learned working as a management consultant at McKinsey’s Bay Area practice.
(15:59) Jason reminisced about his time at the AI Fund startup studio - where he launched AI-enabled SaaS startups by iterating on prototypes, signing design partners, and recruiting the founding team.
(19:25) Jason explained his decision to join the investment team at Greylock Partners.
(22:24) Jason walked through his journey proving value as a new investor.
(24:41) Jason unpacked his checklist for evaluating early-stage enterprise investment opportunities.
(27:09) Jason explained his seed investment in Onehouse - a cloud-native managed lakehouse service that makes data lakes easier, faster, and cheaper.
(30:31 ) Jason explained his Series A investment in Baseten - which builds a powerful software toolkit that empowers technical data science teams to serve, integrate, design, and ship their custom ML models efficiently.
(33:23) Jason touched on advice for his portfolio companies in hiring decisions and navigating product/GTM strategy.
(37:00) Jason unpacked key takeaways from Greylock’s Castles in the Cloud project.
(39:58) Jason dissected key trends in the markets of security, AI/ML, management and governance, and edge computing (as shown in "VC Funding for the Cloud").
(46:24) Jason elaborated on his vision of "The Next Cloud Data Platform" - which examines how the data warehouse, lakehouse, and semantic layer could combine to create a platform for data applications.
(50:55) Jason shared a few books that have greatly influenced his life.
(52:22) Closing segment.
Jason's Contact Info
Mentioned Content
Books
"Moneyball" (by Michael Lewis)
"Why The West Rules For Now" (by Ian Morris)
"Snow Crash" (by Neil Stephenson)
"Cryptonomicon" (by Neil Stephenson)
"Termination Shock" (by Neil Stephenson)
"Principles for Dealing with the Changing World Order" (by Ray Dalio)
People
David Luan (Founder and CEO of Adept)
Alex Ratner (Co-Founder and CEO of Snorkel AI)
Frank Slootman (CEO of Snowflake)
Clement Delangue (Co-Founder and CEO of HuggingFace)
Notes
My conversation with Jason was recorded back in late 2022. Since then, I recommend checking out these resources:
His blog post on the next platform opportunity in cybersecurity
Greylock's investment in LlamaIndex
About the show
Datacast features long-form, in-depth conversations with practitioners and researchers in the data community to walk through their professional journeys and unpack the lessons learned along the way. I invite guests coming from a wide range of career paths — from scientists and analysts to founders and investors — to analyze the case for using data in the real world and extract their mental models (“the WHY and the HOW”) behind their pursuits. Hopefully, these conversations can serve as valuable tools for early-stage data professionals as they navigate their own careers in the exciting data universe.
Datacast is produced and edited by James Le. For inquiries about sponsoring the podcast, email khanhle.1013@gmail.com.
Subscribe by searching for Datacast wherever you get podcasts, or click one of the links below:
If you’re new, see the podcast homepage for the most recent episodes to listen to, or browse the full guest list.
Key Takeaways
Here are the highlights from my conversation with Jason:
On His Upbringing
I grew up in Marin County, located across the Golden Gate Bridge. It's a beautiful place to grow up, offering amazing opportunities for hiking, biking, sailing, swimming, and other outdoor activities. I visit as often as I can, as it has fostered my love for the Bay Area.
Living here has given me a deep appreciation for the local sports teams, and I plan to continue residing in the Bay Area for as long as possible. Growing up, I was particularly interested in baseball and was a true fanatic. With 162 games a year, I would watch at least 150 of them, making it a significant time commitment. While it may not have been the most efficient use of my time, it led me to seek a better understanding of the sport.
One pivotal moment was when I read the book "Moneyball," which was gaining popularity at the time (before the movie was released). With the Oakland A's nearby, the book's contents resonated with me, especially since my favorite team, the Giants, was not performing well. I was fascinated by how the A's, despite having limited resources, could exploit inefficiencies and compete effectively. This sparked my interest in math and statistics, which later evolved into a passion for machine learning during my college years.
On His Obsession with Baseball
In terms of sports, they do an excellent job with youth leagues, especially Little League, which is widespread in northern California. There is also the social aspect and the opportunity to meet people through sports. I was actually born in South Carolina, where there was no major league team. So when I moved here, I was drawn to the Giants because of Barry Bonds, who was an exceptional player. Witnessing his dominance captivated me, particularly compared to other sports at the time.
From an analytics perspective, baseball was at the forefront of that revolution, partly because the game is structured in a very discrete manner. In baseball, one pitcher throws the ball to one batter, who either hits or does not hit the ball, often resulting in one fielder making a play. This makes it much easier to analyze using the available tools at that time, compared to sports like soccer or American football, which are both very fluid games with multiple positions requiring coordination and teamwork.
On His Academic Experience at Stanford
Stanford University is an amazing institution. I feel very fortunate to have attended and formed lifelong friendships there. Interestingly, my wife Victoria was also in my class at Stanford.
Attending Stanford dramatically changed my life. Despite my initial interest in math and statistics, I took a more liberal arts approach academically. This was evident in my studies of Greco-Roman and Egyptian history, culture, and language, and how they have influenced our present day.
Even in the technical realm, my major, Mathematical and Computational Science (MCS), had a liberal arts focus. I took foundational technical courses in computer science, mathematics, statistics, and what Stanford calls "management science and engineering," an evolution of operations research.
I pursued this path to gain a broad technical background in various fields. For my master's degree, I specialized in statistics, which further deepened my interest in data and machine learning (ML). Studying statistics was particularly useful in shaping my perspective on the world and current events. Firstly, it taught me to think probabilistically and consider many things as a distribution rather than fixed certainties. Secondly, it introduced me to a Bayesian approach, where I incorporate new information and adjust my prior beliefs. This mirrors how humans naturally process information, but seeing it expressed mathematically reinforces the importance of staying open to new insights.
We will discuss investments in more detail later. However, it is important to note that investments are primarily about the people involved. Gathering more information on this aspect is crucial. When considering investments, it is important to adopt a people-centric approach. This becomes particularly evident when thinking about portfolio construction. As a venture capitalist, you end up with a portfolio of companies in a multistage firm like Greylock or even seed funds.
When examining what drives VC returns, it follows the classic power law, where certain outcomes and successes are not always predictable in advance. It is helpful to keep this in mind when making investments and to look for opportunities with the potential for significant returns.
On His Extracurricular Activities at Stanford
Mayfield is a fantastic program. I highly recommend that juniors or seniors at Stanford apply. It's a nine-month program structured as follows:
The first three months are dedicated to business school-style case studies. Tech leaders come in to review the studies with you, allowing you to learn from their decisions in early-stage and high-growth startups. During this time, you also form close bonds with your classmates.
The next three months are spent interning at a startup. In my case, it was Sitch, the mobile app company you mentioned.
The final three months involve rotating to teach classes on a topic of our choice. During my internship, I had a fantastic mentorship experience with the CEO.
One of the key learnings for me was the difficulty of consumer-focused ventures. This is why I am now an enterprise investor. Understanding what resonates with consumers and drives adoption can be challenging. However, there are individuals with better instincts in this area.
Using data to A/B test and analyze trends has been invaluable. For example, we discovered that our assumption about the lack of interest in tech events among our user base was incorrect. By adding an element of randomness to push these events, we received a strong positive response.
This example highlights the importance of using data to validate hypotheses about consumer behavior. My time at Mayfield was a great summer experience, and I had the opportunity to teach courses upon my return.
Mayfield also has an amazing community. Josh Reeves from Gusto and Ian Wong from Opendoor were mentors of mine. On the VC side, two of my partners at Greylock, Saam and Josh, were also Mayfield fellows in different years. Being part of this community has greatly contributed to my career.
On Doing Biz Ops at Opendoor
I joined Opendoor during a period of rapid growth, which made it an exciting time. One of the most important lessons I learned about building company culture is the value of in-person interactions, particularly when there are geographic or background differences involved.
While I was based in the corporate office in San Francisco, much of the crucial work for the company was being carried out by real estate professionals in the field. Our initial markets included Scottsdale, Arizona, Phoenix, and Dallas, among others.
My role was a cross-functional strategic one that combined management consulting skills with data science expertise. For instance, I focused on improving our buying practices. While data analysis provides insights, it is not sufficient on its own.
To truly understand the challenges and opportunities, we needed to engage with the real estate leads on the ground. This led to multiple market visits, which provided invaluable insights into aligning our data findings with their firsthand experiences.
By bridging these two perspectives, we were able to make meaningful improvements to our operational practices.
On Management Consulting
The biggest learning for me in relation to my current career has been working with many large enterprises. It has allowed me to see how messy their acquisition processes are, especially when it comes to the acquisition of vendors and software. Additionally, it has given me firsthand experience of the absolute mess and difficulty in managing data, which is contrary to the idealized view of how data should be organized. This is particularly true for large enterprises that have been established for decades and are not in the tech industry.
Observing the messy and disjointed nature of these enterprises and the challenges they face in digitally transforming and improving their data stack has been an eye-opening experience. It has greatly influenced my understanding of the importance of making data investments.
On Launching SaaS Startups at the AI Fund
AI Fund was a great experience for me. I joined early on, and it was an amazing experiment in systematizing the entrepreneurial process and launching companies repeatedly. This is a very challenging task. I was particularly impressed by startup studios that excel in this area. Starting one successful startup is already difficult, but doing it repeatedly as part of a system is even more challenging.
My role was closest to a product role. We had an in-house engineering team, and my responsibility was to identify interesting markets, develop a prototype product, show it to potential customers, and establish design partnerships. If everything went well, we would bring in an external founding team to take over the company and run it.
We explored various areas, including horizontal ML tools like explainability. One of the companies I worked on is still operating successfully, and we also focused on vertical applications. For example, I worked on ML for maritime shipping, even though I had no prior knowledge of the industry before joining AI Fund. I had the opportunity to collaborate with companies like Maersk in Copenhagen and large Japanese shipping giants like Mitsui. The company we worked with, Bearing AI, was recognized as one of CB Insights' top 100 AI companies, thanks to its founders. We also dedicated a significant amount of time to ag tech and explored various verticals.
One common challenge we faced was working in industries like maritime shipping, which were very different from companies like Netflix. While these companies had heard of AI in 2017 and 2018, they lacked in-house expertise. Therefore, we had to figure out how to work with their data and translate our technology, particularly deep learning, which is not always easily understandable, to meet their business needs. The biggest challenge in establishing design partnerships was communication.
We were fortunate to have Andrew Ng's connections and our LP (Limited Partner) base, which included not only VCs like Sequoia and Greylock but also strategic partners like Samsung. These partnerships allowed us to connect with different business units and learn from them.
On Joining Greylock Partners
The opportunity came about as a result of having met several people from Greylock over the years and being very impressed by them.
In particular, Saam was a Mayfield fellow ahead of me, so I knew him from Stanford and met several other people along the way. Obviously, part of being a VC is meeting younger people coming out of university, and Greylock did a great job engaging with me on that front. When we started talking, it was clear that we shared an interest in machine learning, security, and several parts of the enterprise domain. I was looking to move from a startup studio to Greylock.
What was really special about Greylock was the ethos of partnering closely with founders. Greylock makes comparatively few investments as an early-stage fund relative to our peers. Each investor only does about one to two investments per year on average. When we sign up, it's about working closely with founders, putting on the product hat, and collaborating to build use case definitions.
We also get involved in hiring the right people at later stages. The senior partners of Greylock have extensive experience in scaling go-to-market teams and best practices, so they can provide valuable assistance and be trusted partners throughout the company's journey. This aspect was really appealing to me.
Working with some of Greylock's outstanding partners, as well as world-class founders, felt like a good next step in my career. Another thing that interested me was Greylock's extremely successful incubation practice. Companies like Palo Alto Networks, Workday, and more recently, Abnormal Security, all started in Greylock's office. They have become large public companies or fast-growing private security companies. Being able to stay involved in that and provide continuity from my work at AI Fund was also important to me.
Although I was enjoying my time at AI Fund and wasn't actively looking to move, I thought that Greylock, in addition to the incubation side, would give me a more traditional venture standpoint while still partnering with founders. It allowed me to better understand which markets we should be targeting, which was a world that was a little bit more foreign to me at the time.
On Proving Value As A New Investor
Building trust is crucial, not only with other investors at Greylock but particularly with founders. The most significant aspect I am grateful for investing in early on is working closely with founders. This includes learning from them in the early stages and providing value in any way possible. Building trust with founders creates a positive feedback loop.
As investors, when you gain founders' trust, you become more comfortable being brought into other deals and working with companies. This allows you to expand your network of investors outside of Greylock, as well as domain experts and potential buyers. A valuable practice I engage in weekly is reaching out to CISOs of growth stage and large enterprise companies in the security field to understand their priorities and what they seek. This helps shape my investment thesis by gathering first-hand market data and combining it with meetings with the best companies in the market.
By synthesizing these trends, you can identify patterns and make informed decisions. Additionally, as an early VC, it is important to focus on your investment judgment. Don't simply push your own investments or chase the latest hype without being able to articulate why it is interesting. It is also crucial to support other investors at both your level and the partnership level. Collaborating with others not only demonstrates that you are not solely self-interested but also provides valuable learning opportunities.
On His Investment Checklist
When it comes to investments, there are three key areas to consider in the early stage: product, market, and, most importantly, the founders.
On the product side, we thoroughly examine the architecture and technical aspects of what they are building, looking for a unique approach compared to other offerings in the market.
On the market side, we consider existing spending that can be captured, which is relatively more straightforward. For newer markets, we assess both qualitatively by talking to top companies in the field and quantitatively by examining the growth rate of people using new technologies, among other factors.
Founders, in my opinion, are the most important and often the most challenging aspect to evaluate. Some key qualities I look for are their ability to effectively communicate the company's vision and sell it to potential customers, as well as their skill in recruiting technical and non-technical talent.
Having a strong technical background is certainly preferable, as it often leads to better outcomes in technical recruitment. Additionally, a willingness to move quickly and achieve results in the first six months to a year is crucial in establishing a solid foundation for future growth.
Moving swiftly can be difficult to assess based on limited data points. That is why we make an effort to establish a relationship with founders early on, even before they start a company. This allows us to gather more information and insights on both sides – understanding how the founder operates and giving them a chance to gauge what it would be like to work with us as a venture capitalist.
Ultimately, it is essential to find a good mutual match between the founder and the VC, and building a relationship early on helps in making a more informed decision.
On Investing in Onehouse
Starting with the founding team, Vinoth is an amazing technical talent. We have had conversations with him and heard from mutual connections in our network, all of whom confirm his exceptional skills. While at Uber, Vinoth proved to be a hundred times more talented than the average technical professional. He played a crucial role in building some of the most important systems at Uber.
One particular challenge he faced was working across data warehouses and data lakes. While warehouses are excellent for processing and transforming structured data for analytics, they can be costly and have scalability limitations. On the other hand, data lakes, which have fallen out of favor in recent years, are great for managing large volumes of mixed or unstructured data. However, they can be difficult to manage and navigate unless you have expertise in data engineering or data science.
Vinoth, being the talented individual he is, needed a solution that combined the performance of a warehouse with the scalability of a data lake in real time. This led him to create Apache Hudi.
Vinoth implemented a new architecture where the core warehouse and database functionality were directly added to the data lake. He was one of the earliest pioneers of this technology, now known as the lake house. What exactly is a lake house? It's a unified store that reduces engineering time and effort spent on maintaining both the data warehouse and the data lake.
It serves as a single source for workloads across data science, machine learning, SQL, and analytics, which reduces redundancy. With a lake house, you no longer need to run two separate systems or move data across different tools. Additionally, a lake house provides direct access to data, reducing staleness and latency. Moreover, it is a more cost-effective solution compared to maintaining separate systems or relying solely on a data warehouse.
This brings us to Onehouse. Onehouse leverages Apache Hudi and offers a cloud-native managed lake house service. It allows you to build an open data ecosystem. If you think of Snowflake as the pioneer in separating compute and storage, Onehouse takes it a step further at the vendor level. You can choose your own query engine and use it with your lake house. In fact, you can use multiple query engines depending on the workload.
In our investment process, we aim to engage with leading companies and gauge their interest in emerging technologies. We were pleasantly surprised to discover that many leading Silicon Valley tech companies not only had early users of the lake house architecture but were also migrating more and more of their data stack towards it. This trend clearly indicates the long-term direction the market will be taking.
On Investing in Baseten
Greylock had already led the seed in Baseten before I joined, and then we doubled down and led to Series A. This is something we really try to focus on: our seed to Series A conversion. We take pride in our focused approach of investing in seeds and converting them to Series A. It was great to work with Tuhin and the rest of his team from the early stages, and we continue to be excited to lead the next round and build that conviction.
Regarding Baseten, the problem statement really resonated with me from my experience in data and machine learning. The key insight here is that there were too many handoffs between the ML team, data engineering, platform team, product engineering, and users. The whole process of building and operationalizing ML models was too slow.
Baseten came with a thesis of wanting to collapse this cycle and enable faster iteration, putting models to good use within an organization. We have seen tremendous potential for ML in the enterprise, but operationalizing machine learning requires coordination across teams, skilled engineering, and product work from data science.
In conversations with large enterprises, many were disappointed with the ROI they had received from investing in machine learning. Baseten solves not only the deployment side but also makes models easy to integrate into critical business processes and accessible to domain experts within business units. It does this by surrounding the models with business logic and providing a front-end UI. For example, when making a decision on a loan, the ML model may approve or deny the loan, but the loan approval process is complex. You need branching logic to determine if more information is required, if it needs further approval, or if a denial email should be sent. Building this into one product, combined with a front-end UI, empowers business users to interact with the model without using Jupyter Notebook or Python.
On the founder side of your question, they had personally experienced this problem at their previous startup, and they felt like they were the right people to solve it.
On Giving Advice to Portfolio Companies
This is a part of my job that I enjoy the most. The investment team at Greylock is fortunate to receive support from four specialist teams that do an amazing job and spend just as much time working with the founders as we do.
We have an engineering recruiting team that handles product engineering and design, and they also directly hire engineers, which sets us apart from other VC funds that usually only provide advice on hiring. We actually go out and place engineers for the startups we work with. In some cases, we have placed 10 out of the first 15 engineers for a company, which is a significant advantage in a competitive talent market.
Secondly, we have an executive recruiting team that builds a network and pool of talented executives, ranging from CFOs to CTOs and VPs of engineering. This team supports startups at later stages as they continue to scale.
Additionally, we have a marketing and PR team that helps companies with their product launches and communication efforts.
Lastly, we have a customer development team that assists us in maintaining relationships with buyers and CXOs across various organizations.
I must also mention that I couldn't do my job without the support of the broader organization within Greylock. On the hiring side, I always advise startups to look for talented individuals who are at an inflection point in their careers and are ready to grow alongside the startup. While some candidates may have prior experience in the role they are being asked to perform, in other cases, it's important to look for early signs of potential and hire individuals who can figure things out and adapt along the way. As the saying goes, "Build the airplane on the way down."
When it comes to the product, it is one of the most challenging aspects, especially for very technical founders. It is crucial to focus on engaging with as many potential buyers and personas as possible and clearly articulate your vision, the problem you are solving, how you are solving it, and who you are solving it for. Many companies make the mistake of rushing through this stage and failing to establish a solid foundation, which can lead to problems down the road, sometimes not realizing it until Series B funding.
I would emphasize the importance of developing a Go-to-Market strategy and hiring the right people accordingly. The hiring process should consider both top-down and bottom-up approaches, as they require different strategies. Additionally, pricing is a critical factor that is often overlooked in the early stages. Many companies either overprice or underprice their products, which can result in setting up the wrong incentives based on how they price their offerings.
On Castles in the Cloud Project
This is an undertaking that I did with my partners, Jerry and Corinne. The three of us set out to build a comprehensive project to map the cloud ecosystem. As you mentioned, we launched in 2021 and made an update this year. We intend this to continue as an annual project, analyzing the trends of how the cloud ecosystem is evolving year by year.
We also want founders to submit companies to be included in our analysis. Some have even unintentionally benefited by getting sales leads from being part of it. It also gives them a chance to understand the market size, competition, and opportunities.
We hope it becomes a participatory exercise as well. We categorized every cloud service offered by AWS, Azure, and GCP into markets and submarkets. Then, we analyzed where startups compete against them and their success rates.
There are around six or seven hundred cloud services out there, with hundreds of startups in our database. Some key takeaways we discussed on how startups can successfully compete against the big three are owning the community and having passionate developers, implementing a product-led growth (PLG) motion to reach users and personas beyond cloud architects and engineers, and building deep intellectual property (IP) like Snowflake did with operational excellence. Snowflake, for example, rethought the entire data warehouse process by separating storage from compute, taking advantage of cloud computing's elastic resources to build a better database.
There are other ways to compete against the clouds, but I'll leave it there.
On VC Funding for the Cloud
I love all these markets. Let's focus on a few in particular. Starting with security, the cloud offers amazing benefits but also presents increased security risks. The potential surface area for attacks is larger, and it operates outside the safety of controlled on-premises systems. Traditional security teams and vendors often have a limited understanding of cloud security.
There's a notable statistic: the number of cyber attacks per week on corporate networks increased by 50 percent in 2021 compared to 2020, reaching an all-time high in December due to log4j. Greylock has achieved significant success in this market and remains optimistic.
In an uncertain economic future, cybersecurity is considered a necessary expenditure rather than discretionary spending on software. As the market declines, hacker and attacker behavior tends to increase. An interesting trend is the intersection of cybersecurity with data and machine learning.
There is a shortage of cybersecurity talent, with up to a million unfilled roles or jobs in the U.S. alone. This creates a real need for automation and machine learning to identify potential attacks in log data and prioritize vulnerabilities detected by various tools.
Moving on, another area of interest is machine learning. Over the past year, large language models have been a focus for us and the broader investor market. This includes building large models and developing applications based on platforms like GPT-3. We have seen the emergence of a new generation of companies in this space.
Large language models offer an intriguing opportunity for collaboration between systems and humans. The efficiency of software engineers using GitHub co-pilot, for example, is remarkable compared to working alone. Co-pilot generates or assists in writing significant amounts of code.
Greylock has made two significant investments in this area. One is Adept AI, and the other is Inflection, co-founded by Reid Hoffman and Mustafa, the former co-founder of DeepMind. Both companies work on large language models, with Adept AI focusing on the enterprise side and showcasing some impressive prototypes.
Next, I want to highlight management and governance, including ESG (Environmental, Social, and Governance) initiatives. There is a significant opportunity to help enterprises become more sustainable and manage their sustainability teams. Several startups in this space focus on cloud-related data challenges, such as gathering emissions factors across complex organizations. Another important trend is cloud cost management, as highlighted in Flexera's 2022 State of the Cloud report. Respondents reported being more than 10 percent over budget, with about a third of cloud spending estimated as wasted.
Greylock has invested in cost management, including Cribl, which provides a neutral routing and filtering layer for different APM and observability tools. We also recently announced Bluesky, a solution for controlling costs in platforms like Snowflake and Databricks.
Lastly, we acknowledge the significance of edge computing and IoT. While we have not made as many investments in this area yet, we recognize the growing importance of storage and compute on the edge. Use cases include AI gaming and the deployment of WebAssembly (Wasm) on the edge. We closely monitor the technology, both within and outside the browser, and have observed its power in applications like Figma.
Determining how to allocate time in VC is often a challenging task. We make an effort to speak with IT and computing buyers across large organizations to gain valuable insights. We also rely on qualitative findings, such as those from Gartner and other publications, and combine them with our in-depth analysis of the startup landscape to identify exciting opportunities and revenue potential against major cloud providers.
On The Next Cloud Data Platform
This is an area I'm really excited about. As discussed earlier, Snowflake has been one of the most successful stories. It competes against the cloud based on deep IP, and a Snowflake ecosystem has sprung up around it for data management tasks such as data input, quality checking, and data output.
The evolution of this will be data apps, which are internal and external apps built on Snowflake as a backend. Security companies like Lacework, Hunters, and Panther offer the option to use Snowflake as a backend data store. Snowflake's acquisition of Streamlit could be part of an effort to develop a Snowflake app store.
Looking ahead, I think lake houses could play a role. They provide a single ecosystem that enables real-time streaming, machine learning, and data science use cases, which are important capabilities for building comprehensive data apps.
Another component, as you mentioned, is the semantic layer. Regardless of whether you have a lake house or a cloud data warehouse at the core, a semantic layer is essential. It standardizes metric definitions and business logic independently from downstream systems like BI tools and ML notebooks. This ensures consistent definitions across the company and prevents disruptions in downstream systems when changes are made to the core data store.
The semantic layer goes beyond metrics and creates an API for people and applications to directly access business objects tied to the lake house or cloud data warehouse backend. This empowers analysts to engage directly with concepts like customers, plans, and products and allows applications to be built on a more stable foundation instead of relying on custom SQL queries that can change and lead to data store disruptions.
In the long term, these applications could even have the ability to write back to the database, creating a transactional experience and replicating some of the OLTP functionality. Combining a more mature development base with simple, scalable, and unified cloud data warehouses and lake houses helps realize the vision of a cloud data platform. It challenges the platform dominance of the three major cloud providers at the developer level.
Share this post