Building Data Products - the Do's and Don'ts - Yulia Trakhtenberg, Gloat & Victoria Kalmanovich

In our first episode in Season 2 of Dev Interrupted - the Hebrew Edition, Yishai Beeri, CTO at LinearB, hosts two guests from Gloat - Yulia Trakhtenberg VP R&D, and Victoria Kalmanovich, Backend & Data Engineering Manager (as well as DevOpsDays Tel Aviv musical performer). In this episode they chat about the unique challenges of building data products, software engineering with a focus on data aspects, and of course how this impacts engineering management and developer experience...our favorite topics.

Episode Transcript תמליל הפרק

Hebrew, then English* בעברית ואז אנגלית:

(*Translated with Google Translate - so there may be some errors)

Yishai:         Welcome to Season 2 of Dev Interrupted - the Hebrew Edition, LinearB's podcast which discusses everything that interests the daily work of engineering managers. This season we will host industry leaders and talk with them about everything that interests engineering managers, those who work with them, and those who one day would like to manage an engineering organization. This season we will place special emphasis on Developer Experience. I'm Yishai Beeri, CTO at LinearB. Let's get started.

Yishai: In this episode, I am happy to host Yulia Trakhtenberg, VP R&D at Gloat, and Vicky Kalmanovich, engineering manager at Gloat. Hello, how nice of you to come.

Yulia & Vicki: What fun to be here.

Yishai: As always, we start with just a few words, we would love to hear from each of you how your career developed and how you got to this point, so Yulia, let's get started.

Yulia: I'm starting, okay. So I've been in high-tech I think for over 20 years, I did a degree in computer science, like everyone else, I started in the times of the first crisis, 2001, in telephony, where I learned very well C++, the depth of threading and all the bits & bytes of development. From there I evolved into management, I started as a team leader, a group leader, and I realized that the world of startups is more interesting to me and I moved to work in startups. So for the last two years I've been at Gloat, engineering manager. In the last two years we have grown significantly, both as a company and as a development, we have developed new products, we have changed the direction completely, we have brought in very significant enterprise customers, so I think the path we have taken as a company is very interesting and we will talk about it a bit.

Yishai: Definitely. So today you are VP R&D at Gloat, how many people in the development organization more or less?

Yulia: 70 people.

Yishai: 70, great, so we'll dive right in to hear a little bit about what Gloat is doing, but Vicky, come on, tell me about your track.

Vicki: So I've been at Gloat for a month and a half (laughs), which is very exciting.

Yishai: That's it, "Grace" is over now.

Vicki: Yes (laughing) in startups it ends very early, you know that. So before that I was at a company called Aspectiva, which was acquired by Walmart, we also made all kinds of data products based on NLP there, and before that I was about 10 years in technological positions in the army, so I've been managing for almost 7 years, and that's it.

Yishai: Excellent. So tell me in a few words what Gloat does, just so we have context.

Yulia: Gloat develops products for enterprises in the world of HR Tech, in fact organizations today really, really want to know what skill sets their workforce, their organization, their employees have, and plan it forward. What will be missing in a year? In 3 years, in 5 years? And actually we are developing several products that give one, the ability to understand where the organization stands now compared to the market, compared to its vertical and where we want to take the organization, and this connects to the other product that allows employees within the organization to develop. So basically there are two sides here, first, how the employees develop, what opportunities they can take and develop their career within the organization, and second, how the organization can also direct them to something that will be very relevant for the organization in 3 and 5 years.

Yishai: I understand, so it connects to systems, say like workday and those of more HR management, and on the other hand to learning and development?

Yulia: So we are actually standing in the middle, we connect to systems like workday, HCMs of all kinds to get the data for the employees, to know what they know how to do, what skill set they currently have. Of course, they can enrich it within our platform, and connect to systems like learning and others, which make it possible to bring to the talent marketplace, which is actually the second product I said where the employees develop, a lot of opportunities, and where the employee can say I am a junior frontend developer today, I see myself in 5-6 years senior architect. How do I do this within my organization? And with the help of all the opportunities, mentors, courses, projects, jobs in the middle, we build his career and help him take every step in his career and explain to him what he needs to do.

Yishai: I understand, so it also connects to the world of hiring and promotions or skilling up, in the end I was able to advance in my skills, maybe I can get a promotion and advance.

Yulia: Completely.

Yishai: Excellent. Today we will talk about data products, some kind of world of products that in the end uses data and makes data available to customers to give them value, and Gloat sounds like a data product. And I want to touch with you about, as if to understand where it is different, where it is similar, what challenges it poses to a development manager or a team leader or whoever is ultimately involved in creating it. So Vicki, maybe we'll start with you, what do you think, where are the main points that data products behave differently.

Vicki: Me, the truth is I'll maybe start a little to connect with what Yulia said before about the product. In general, many of the things we do today, they are supposed to help many people within very, very large companies, that it is very difficult to reach the right opportunities for each person in an individual way without looking at things in a very general way, as if, and using data. There are companies that have like 100 thousand employees or more, and it is very, very difficult, I think, for HR organizations within these companies to reach each employee and help each employee individually. And so here data is simple, we must be a data driven organization, there's nothing you can do. You will simply never be able to reach your people and if this is something that is important to you then you must learn how to understand the employee, how to get all the necessary data about them and how to use this data later to give them the most suitable opportunities for them. And how is it different from other organizations? It's simple in this aspect of what you do with this data, like how you collect the data, what you do with it, how you give the insights back.

Yishai: Yes, so if I look at products, in the world of SaaS, or in the world of software products in general, there are many things related to flow, how I do a task, how I achieve a goal, and if I look at data products, then perhaps the data is The king and not the flow. It doesn't matter either, it's certainly important that the UX is good and that I can do something with the data, but first of all there is value in the fact that this data is displayed, there are insights, etc. And I'm interested to understand, when I develop such a data product, where the emphasis is not always on the usability or on bringing me from place X, I don't know, I took "Lime" on the way here. There is a lot of data there, but it doesn't serve me as a user, I have to find the scooter in the end, turn it on, get here, close the trip and pay. So for me it's not a data product, there are other investors,

Vicki: You don't, but "Lime" also has a lot of data, for example to know, to bring the scooters to the places that will need them the most. So I think that today when we live in the world of products with a lot of users, almost every product has some kind of data component and probably big data, yes? To bring the insight and to bring the product, this is one style of the products. And the second style, I think that today the world of go to market is often based on the data, when you want to sell, when you want to decide which product you are going for, in which direction, we have endless options and we have a short blanket of resources. We always want to know how to prioritize and all the, once these worlds of risk and effort and all these worlds of deciding a product are nice, but today there is another component of data analysis that you can do, to say in advance whether the product is going to be successful for you or not, according to The data you collect from the field before you redeem the product.

Yishai: Yes, so there is a world of data for internal use, data for BI, data for product analytics, to reduce friction in the product, to understand which users, what they get stuck on and so on. And there are the data products that actually put the data mainly for the blokes and users who end up using it.

Vicki: When you take data analysis as if, I will connect for a moment to what you said, if we want to do, you said before that there are, say, tasks in the classical world without data involved. But if we want to know what value we actually want to give to the users, and we have a lot of data and we just have to think about the right aggregation or what not, what is the right processing I need to do to the data, then I can do it in advance. I have all the data, I can analyze it, I can check what features I can provide that will give what kind of value and then according to my processing of the data I can understand where my users will get their maximum value.

Yishai: Yes, which is actually internal use.

Vicki: Both internal and external use. What is meant by internal use?

Yishai: If I have to decide how to develop a feature or even what UX is appropriate, I use the data I've collected to develop a better product. The interface I was working with that I took the scooter to here relies on data behind the scenes, someone said voila, if the button is on the top right, the users are less confused.

Vicki: So my intention is more, for example, if the company as a data products company wants to give some kind of product to the user, and it does not know if this product will be valuable based on the data it has, then it can be done with the tools we have / like we can do analysis on the data we have, to understand How do we do this processing and it's not for internal use, it's more to understand what the customers need and how we answer it.

Yishai: Okay, so when I build a product that again, the data is its heart and it is also the main value that the customers get from it, where is the development process or the work process in building this product? Where is it different or similar to the process that I build a different type of product, which is not necessarily data oriented or where the data is not its main point, where does it meet me in cycle processes versus a product or in learning versus what works for customers and what doesn't.

Yulia: Me?

Yishai: Go for it.

Yulia: So I think that if we start talking about product development, I think that many times every product manager, every company, will prefer to develop a product that has some kind of real use case and even a partnership, a design partner, with which the product can be developed. And this also applies to products that are data oriented and other products. In products that are data oriented, I think that when you have a design partner who can look at the data and say how high-quality it is, how much it brings value to it, in the end what are the products, what is all, we introduce maybe some kind of innovation and things like that and we also reduce the amount of manual work And maybe it's tiring to build things, maybe we're solving a problem that can't be solved, as Vicki said today in the world of, in the big companies, you can't go to each and every employee and offer them things. So in these cases, do we offer relevant proposals, do we offer relevant routes to the employees, I think that without a design partner we would not be able to know if the flow, is our product good, because the flow might be fine, yes? We offer a career path to the employee, but is this career path interesting? Will the employees like or dislike him? This is something that we have here a design partner to help with, and this is usually a characteristic of data products, which you look at,

Yishai: This means that their reliance on a design partner is more fundamental and it is more difficult to progress without a real design partner, unlike perhaps other products where I can make a little more progress, even if I don't yet have the exact design partner who gives me feedback at every moment.

Yulia: Unless you know what the end game is, right? What is the final product you are, what is the final impact you want to achieve, yes? In our cases, because it's iterations, because it's working with people, it's very hard to know what people will like and what people will prefer.

Yishai: The data can also surprise us.

Yulia: The data can also surprise us. Every organization is different in its structure and the employee's ability to progress within the organization, so it is very interesting to work with the big design partners who define this field. Basically the whole world of employee development is relatively new.

Yishai: Yes, so the visitors don't always know what they need and want either. Does this mean that when I work with design partners I, again, on data products, I should demand more from them? There are additional skills that they need to be able to look at data that is not yet completely digested and say there is something here. I mean, maybe they should be power users more than regular users, when I do a design partnership on a product that is mainly flow?

Yulia: As with the adoption of any product, you have these crazy early adopters who will go with you even when your product has a thousand bugs, and will show you what's good and what's not good, I think that's how you should also choose the design partner. You probably won't bring someone for whom everything has to be precise and orderly, probably your language, you will probably fail, if you build a design partnership based on such a person or such a client. But if you take the dreamer, and you go with him, then he can dream along with you and say oh, here I see my value, let's start digging here, in this area.

Yishai: Okay. What about delivery of value outside the product, as a learning tool? I can imagine, if I think about your world, let's get you Google Sheet or some kind of Excel with data that is already actionable, but it's not sewn into the flow, it's not in the product, we'll get feedback on it, it seems you're able to do it, let's say if I'm talking about career path, someone can look at this output with their eyes and say voila, it seems logical, voila I'll manually grab 10 employees and try to see their feedback on the data I took out. Completely outside the product or completely, as if much less engineering, to receive the feedback and then continue, as if to advance the feedback from the field, to real development work or closure of flows and UX, etc. It's something you get to,

Vicki: The question is what do you want to achieve with this, as if you want to brand yourself as a company that provides raw data APIs and you do some analysis that suits you as a company? That's also fine, there are companies that do that, or you want to brand yourself and give a lot of tools like we do, which is simply a whole platform of everything from everything, and then give some other raw data that you can analyze yourself, it doesn't give the full impact Like getting everything you get within the entire platform.

Yishai: Yes, so I'm thinking of a separation here between, if I'm talking about, again, some kind of story of career path offers, then I have the, okay, at the end an employee has to receive offers, personalized, and respond to them, choose them and use the system this. So the system is measured by the fact that it's easy to find and it's easy for me to give feedback and understand what I'm seeing, etc., but it also has some core that says did I bring you a good career path or not, I can disconnect the questions and say, I'll put it in document, tell me yes or no, and I still don't ask myself whether the flow is intuitive, whether it's easy to find the button, how to search. As if I am separating usability questions from questions about data, maybe our data does not know how to flood good career paths and then I have nothing to build a UI and a market place and flows if the data does not work.

Yulia: I really connect with this approach, I think that before you start investing a lot, you need to understand that the basis is good, and the basis here is data and an AI-based algorithm that can bring you good data. More than that, you will probably keep improving your algorithm all the time and that will be your way of testing, you won't connect it directly to production and UI to know if your lanes are good, yes? So I really relate to this approach, I think, I can say that even in the organizations I worked in before this, before Gloat. We had a product for a whole year that was run in Excel with design partners, without a UI, without anything, to prove that this product was even feasible, before we brought value to several customers who worked with Excel, we did not build a line of code either in the backend or in the frontend, only in data. Everything we did was just in data, just to see that this product would bring value.

Yishai: What does that do to a product person? Doing such a process is either completely radical outside of the product without code, or the main questions are whether the data is good, whether we were able to bring insights into how the relationship between the product and engineering changes or looks different when that is the focus.

Vicki: I think in my previous company that we also did data products and in fact the product manager that I worked with there she was very strong in metrics and she really tried to understand the customers and what they need and how we produce as if the most valuable data features, as if then she was very, very connected to aspects of the data analysis, for aspects of the metrics we want to find, and also for what AB testing we should do and aspects of the data. So the more the product was really there and connected to the data itself and try to extract from the users what their needs are, it could be very, very successful.

Yishai: And on the other hand, the engineering group that builds it or makes iterations on the data, so sometimes also engineering and data scientists or analysts, they have a greater weight in deciding what is possible, what should be built, what is right, again, compared to a product and compared to products that are not Rather data-oriented.

Yulia: I think the boundaries here blur a bit between product people, development people and whoever checks the data at the end, yes? Because the product is measured not only on its features, but also on the quality of the data you bring. Flows at the beginning are even missing within the, at the beginning of the product, right? There will be no flows. So all we have is data, and data, there should be very, very quick iterations of evaluating the quality of the data, understanding how far we are from the direction, maybe we can make some tweak here and change the algorithm a little more and bring something much better. So if we talk about organizations, HR organizations are very version oriented, there are tests and things like that. In data products this won't work, probably for data products in their beginning, in their development, we will have to make some kind of, several changes a day, and talk 5 times a day in Zoom with the customer to know how he experiences the change in the data we made in terms of the quality of the results.

Vicki: I just want to take a moment to give an example from the first data feature we uploaded to Walmart. I remember that it was very, very hard work by both the engineering, the data scientists, and the product together, and then when we defined it as if, from the data scientists' side, they tried very hard to achieve precision and recall that would be suitable for what we wanted in order to define. That the data was high quality, and at the end there was some kind of "good enough", like okay, we didn't reach the maximum precision and recall. This will be the first iteration, and the engineering from our side we took and tried to understand in terms of how the data, does not know, if there are fields that look strange or there are nulls or all kinds of things like that very very basic, in order to understand whether the data we are providing now for a data feature This is really the best it can be and the most complete it can be. And from the side of the product as well, in terms of bringing the value properly, so it was very, very hard work to try to understand the quality of the first data feature we are bringing, and also to already plan what the next iteration will be.

Yishai: Yes, I think that if we look at, as if from a classic definition, product people write user stories, and talk about whether the user succeeds in performing the task, and in most cases, again, of classic products, it depends on the flow, whether the flow allows the task to be performed in such a way that the user receives the what he wanted When you put data into the picture then the question becomes okay, I can do the task, but the question is whether the data was good enough to do it with confidence or to make a decision, and then it becomes something very fluid, who said the data is good enough, Like how do I know the user was successful? He managed to move the button from A to B, he can technically choose to act on the data, but the question is whether the data is good enough for him to really feel comfortable choosing, and it is already becoming something that is fuzzy.

Yulia: Right, more than that, the fact that he didn't click, the fact that he didn't select, say, the specific line, is it because the user interface was uncomfortable and he couldn't find a button? Or because the line was bad and he didn't want it? Okay? That's something we can't even assess with the standard normal product analytics tools, right?

Yishai: Right, or I'm creating a UI that allows me to give feedback, but if I go back to the story, the story says let him choose a line, ok, the UI allows you to choose a line. And there is not a single word here, or at least it is difficult to insert a moment's speech, that the offers will also be good. And what does favor mean? This is from a new production in Stories, compared to other products.

Yulia: Yes, we are breaking the world of functional or functional here, right, where is the quality of the data? Neither here nor here.

Yishai: Okay.

Yulia: You can tell, right?

Yishai: There is some, right?

Yulia: that suddenly the world of the product introduces some kind of third world, a third dimension into the world of requirements.

Yishai: Yes, if we increased the quality from 82% to 84%, is it functional or is it non-functional, how do I measure.

Vicki: Also how did you define the 82% (laughing) based on what?

Yishai: This is a deep question, but let's say I have a number or let's say even each of the users marks, but it's a thousand becomes a sequence, so what's the extent of a story, who decides that it's good enough, and also wait, is it a functional or non-functional change? The user story didn't change, I just gave a little more value.

Vicki: For this you have all kinds of tools like AB Test. I don't know, do you want to know how your stranger experienced this thing? Let's do an AB Test, let's see, 50% like that, 50% like that and let's see how investors experience.

Yishai: So it's a tool that helps me choose between two versions, but is the story complete?

Vicki: So let's say let's say, I'll go back to the example I gave, we tested the quality of the data, we reached a sufficiently good precision recall, we reached the point where we don't have nulls in any field, everything is good, and the numbers look fine in terms of quantities, let's say. Then we say ok, the quality of the data is good enough for us, to put it on the market for a moment and we will get some kind of opinion from users. Then you can do, say, AB Testing or all kinds of other product tools that I'm not that familiar with. But let's say you can accept it to get some kind of opinion and then see if you need another iteration, maybe you need to download this feature, maybe this feature is not valuable, I don't know.

Yishai: And what moves forward these questions of let's do AB Testing or let's do, I don't know. The quality is not good enough, let's move on, is this a product warranty? at the end? Or responsibility of a group that develops it?

Vicki: After all, we should all be owners of this data and to forge ahead, if I see that the data is not of high enough quality and I am in engineering I should feel comfortable enough to say the data is not of high quality, let's do everything to increase the quality of the data.

"After all, we should all be owners of this data and ...if I see that the data is not of high enough quality, and I am in engineering, I should feel comfortable enough to say the data is not of high enough quality."

Yishai: So there is a bit, let's say, more blurring than usual between product and engineering, in terms of who decides that it is good enough, who is the driver to make an improvement, because it is common to think that engineering wants to promote the non-functional, they want to have code that will allow them to work easily Also about the next feature, but that the product is the ones who finally decide or determine, this is the shell of the story, this is the flow that needs to happen. And in the data, I hear that there is a little more mixing here between what product and engineering have to give an opinion on, whether it is good enough, whether it passes some level of experience.

Yulia: Yes, there's a new role here that suddenly popped up, right? Within R&D, it always existed, but not within R&D, suddenly we have a data analyst that is needed, who may be the one who signs off on the quality of the data, on the completeness of the story, this is something that did not exist before they started working with data products .

Yishai: So let's really talk about the analyst, first of all where does he usually sit? Is it part of the engineering organization? Is it part of the product organization? Is this a third party?

Vicki: In my previous company (laughs) the data analyst sat within the NLP team and worked together with them and it worked amazingly. Because she knew exactly what was in the data, she knew exactly what worked, what didn't work, where there were things that needed to be done, and she was also very verbal in the way she said where the pain was.

Yishai: An NLP investigator...

Vicki: (Laughs) Yes, she was also a linguist, right. So she really knew where the pain was in a very, very clear way, and I think it's a very, very great value for the organization, that there are data scientists who deal with things that are more long-term, and then the data analyst can come and look at the data for a moment and give some sort of situational picture.

Yishai: So you are actually saying that it worked well when the analyst was part of the development, and not of the product, which maybe it was, if I had to pull out I would say it is closer to the product, because you have to say what is good enough and what is not good enough, more about the what and less about the how.

Vicki: Because it depends on what the goal is. There the goal was really to raise things like data quality.

Yishai: Okay.

Yulia: I think there are several layers here, yes? There is some sort of layer of data scientists here who work with their models and when do they say the model is good enough to start putting it into development. Here we need an analyst who will sit within the organization of data scientists, as Vicki said. And there is how this data is already connected to the product, and here I do think it sits somewhere in the product, between product and development, yes? But already in development, not in data scientists, for a team of scientists, with us it is relatively separated and connected. It's impossible to know, it's quite fluid, how data science connects to development, but I think it's two completely different layers of analysis that you need to do.

Yishai: Okay, so we have, a data scientist is something that is a little more research, it is a little more open-ended, they also end up working on product tasks, but maybe they do a little more exploration, not sure the direction they are going He will succeed. But then you say you need a data analyst as part of the development team, living in iterations, living in delivery, and its name is no longer exploration, it's building the value or measuring it as good enough, and here it's not an endless process. And then the downside is what, that a data analyst is assigned to a certain team that makes a certain product and he can't also serve a thousand and one other things? Is that sometimes the reason to put analysts outside? Let them be both this and this and this and BI and,

Yulia: It depends on the size of the organization, yes? I think this is how the organization develops. In the beginning we all have several hats, yes? Each of us does several things. So in a third of a job we do it and in a third here and there it adds up to a little more than 100%, right? (laughing) But as the organization develops you feel more and more the need, this is how I saw it in the previous organizations that developed, there is more and more need to have people dedicated to the specific business, to the specific business line of your products. Because you know them, you understand them, you also know what their goals are and it's no longer possible to focus on eight products, but perhaps you need eight people within one product, to give the real value, yes, it all depends on the scale.

Yishai: So if I have a data analyst in the development team, how is the day to day going? As if developers take tasks from the backlog or from the sprint or whatever and work and create code, the analyst also works on stories? Taking stories is their owner? Or is he a helper or a helper, a companion to the engineers for their tasks?

Vicki: I think it's really like a per-organization, in the previous organization I was in, which we were, it was really one team of data scientists, the organization was very small, there were less than 30 people at that time. And she was part of the sprints and she had her tasks and she had to, just as data scientists now examine some direction to see if it is good or not good, she also examines directions to see if they are good or not good. If now we have some kind of feature that we wanted to run, some kind of recommendation system based on certain data, then it goes through this data, then it makes all kinds of queries on it, then it checks whether the recommendation system will really give the data insights we want will she give?

Yishai: She works on things that are like one spirant forward or backward before the development itself?

Vicki: In a perfect world yes (laughs) but in general it all happens like this probably sometimes together.

Yishai: Okay, I found something good enough, let's put in a task to develop or make it organized in terms of engineering and move forward?

Yulia: Maybe we can give a real example. We are now releasing career path version 2, okay? which is based on a completely different data science algorithm, okay? So basically there was an iteration between engineering and science, to bring all the lines without, as you said, in Excel. And we went through Excel and improved the model and how it connects to engineering, how do you take the things and the graph that was built and where is the limit, where do you cut the coefficients, yes? At 0.6 or at 0.8? Okay? Here we were completely cut off from the customers, we were cut off from the outside, but we used the data we collected from customers and tried to see if our direction was correct at all, if this model was going to be successful.

Yishai: Yes.

Yulia: Today we are at the stage where we have already taken on two clients and are starting to implement this feature in two clients, it is not yet going to be released to everyone. Because we feel that there is also tuning on the data, okay? So maybe we thought that in the ideal world it was going to be like this, but here it is with this client it works like this and with another client it works like this, so maybe we need to do something iterative before the client releases this feature he will decide where he cuts according to some sample ) or something like that, and now it affects the feature. So here too there is an analyst who sits and influences, and here I said, this is exactly the difference between an analyst who checks the model, the data is clean. And how it connects in the end to the product and the data of the specific customer or several customers in order to understand how this model will live within a product.

(transitional music)

Yishai: Okay, so we talked about job holders and the importance of the ability of a data analyst and within the development team, let's really talk a little bit about the skills or how I build a team that knows how to deliver data products. On the hiring side, on the skilling up side, how is it different or, Vicky, how do you feel, what is critical to put your finger on to make sure there are the right people for this task?

Vicki: I think that many organizations in recent years have gone through this shift from systems that are back-end to systems that now need to support a lot of data, and this is a completely different stack and it's also completely different skills. Like if I now save data in a certain way in a back-end system and I just need, I don't know, to transfer, it doesn't matter, I won't give an example. But if I now want to switch to a system that gives me data insights, then I need to learn how to create data pipelines, I need to learn how to work with scheduling certain tasks, I need to work with events. I seem to need to transfer my entire system to a different form, that back-end systems did not work in this way. And a lot of organizations from what I feel like this, from conversations and authors that we talk about these things is very, it's a difficult and non-intuitive transition.

Yishai: And it happens, from what you see, in such a concentrated way? Is there some infrastructure team that builds the pipelines and builds some infrastructure that later product teams can use? Or does everyone do it for themselves?

Vicki: From what I gathered some information (laughing)

Yishai: Who installs the Kafka?

Vicki: (Laughs) So from what I gathered a little bit is that it usually starts as a pain for the organization, then the organization gives it a team to do something small and then they feel that it is needed throughout the organization and then they create some kind of data platform or something for it as if it is a little more inclusive, because the organization understands that he moved to become a data organization. But in order to become a data organization you have to start gathering the people, like starting to bring in data engineers who understand how to work with data, like bringing in Kafka when you're a back-end engineer and you can't expect things Like things that happened to us, like suddenly rebalancing in production when you have insane amounts of data, how do we know these things? We haven't worked with such amounts of scale before, so these things like you should have the right set of skills to work with data, that A profession that is relatively new, so as if there aren't that many people who have experienced these things, on a very, very high scale.

Yishai: So if I am an organization or think of myself as an organization or a team or a product that is data oriented, then a model due to this is already when there is a platform and teams that maintain something central for everyone and an initial model that usually starts with one of the teams that encounters the first need, starts to establish something.

Vicki: Do POC, POC is always good, start from POC (laughing).

Yishai: Okay, but then there is the transition where they say okay, it's not good enough, it's something ad hoc, I need to dedicate a team to it, which will be its day in day out, it's a team that doesn't actually build products, it builds infrastructure . So it's a bit, how is this transition from, I'm interested in the dynamics of the transition, when do you decide that okay, it's time to establish an infrastructure team that will do this and not deal with the delivery of a product.

Yulia: I think when you suddenly recognize that you are duplicating things, yes? Either you suddenly duplicate how you work with Kafka, which you mentioned here, or you suddenly duplicate the data, which is already starting to hurt, because you have discrepancy data and it costs you too much. So how do you build one thing that serves both sides, and how do you not compromise the functionality of either, yes? So basically if it's one team that started and continues to maintain it, then it remembers its own interests and sometimes forgets the interests of the other team and can hurt. And if it's a platform team then it actually has two clients and has to serve both of them, and that's the transition I think you've already made, when it starts to hurt like Vicky said, and the pain can be duplication, discrimination, things like that and less I think of a desire to put something in the center when there's still not enough ,

Yishai: Can we even start with something central? Is it realistic? This is true? I'm fine, I've already been in several startups, I've been through the pain, let's start with the Tim platform that will put it in the center. Can't skip this dirty and initial step? Or that, I'll make mistakes, it's just not right, until you experience the pain and then move.

Vicki: Always experience the pain and then move, always, as if it's always true, this, if you want to do, it's like-, I don't know, we all want to do like Netflix and like Twitter before the era of what's happening now, but we're not in the same needs , we are not in the same pains, so as if let's understand what our needs are and what our pains are and then we will build what we need.

Yishai: Okay, so I think it's interesting as even someone who is being interviewed at a company or who looks at a company for a moment from the side, to understand where he is in, where this company is in the process of an ad hoc solution to data problems or the establishment of data needs for a specific product, and the situation that has I have some kind of service and bus and something central that all the data products and maybe not the data products rely on. Great, like a diagram for architecture, but you can't get there straight away, you have to start with things next to it and connect. What does it do to a team that stops devising a product or is only talking about an internal platform, as if it finally has one, this is a very big shift in how it is conducted in the discovery of value and feedback on value, and does not deliver user stories at all. It's not for everyone, so what does the transition look like assuming you're not completely bringing in new people to build your platform team or data team. How do you find who is suitable for this and what does this change look like in terms of the life of, even team lead who until now delivered products and now he is delivering a platform.

Vicki: I wouldn't say that you don't have a shift in people here, you must have a shift in people here because if I'm a team leader who likes to see so much, count the page views or things like that, I won't, I probably won't find myself on the platform, yes? More than that, it's probably a team job without a product manager, because what's a product anymore, right? There are no more product definitions, no customer facing, it starts to be only technical requirements.

Yishai: Right.

Vicki: So I think that when you do something like this you naturally, with the growth of the team, you naturally see which people are more attracted to the product, to something interactive, versus working with customers, working with the requirements of colleagues, and something that is perhaps a little longer in the cycle, A bit more of a service provider for other teams, yes? And here I think it's usually different people, it's not the same person who can suddenly make a switch. There are some who do like it, but mostly I've seen it start within product teams, some people like customer facing more, and there are more people who like it more the infra within the teams, and as the teams grow and the organization grows and you already decide to make a platform group, then you move these people in this direction and create a group here.

Yishai: What happens when I work on a product that is data-based and perhaps relies on some platform, and I don't get a fast enough or flexible enough service from the Tim platform. But I need to think and I want, I don't want to be stuck, how, like,

Vicki: You moved to the next step, after we already did the, we built a platform team, now this is when it doesn't work well (laughs)

Yishai: Or when it's great but I have something, I have needs that may not be the highest priority within the organization.

Vicki: And there is no product, you say, so there is no one to prioritize?

Yishai: And I need the data like this and it will take them six months. What, how do you solve this? As if one way I know is fine, I will solve it myself, we will do what is necessary ad hoc, but I have now created a new pain.

Vicki: I think the platform should have an enablement and not a bottleneck, but it's like a philosophy already (laughing)

Yishai: The question is if in real life, do you see it? It happens?

Yulia: I believe that the way you started from the pain and then reached you reached platform engineering, here you are too, here you are in the iteration. So you created a new pain, when it hurts enough, put it back into the platform. Do something yourself, run with it, prove that it's good, prove that it's important, when there will be other teams that will use it and it will be a real pain.

Yishai: Then force the platform to eat it.

Yulia: In the natural way it will enter the platform, each iteration.

Yishai: Okay, I'm sure the people of platform teams really like the sentence you just said (everyone laughs) and say yes, everyone was given permission to do darkOps, but maybe that's the way to deal.

(transitional music)

Yishai: Let's talk about such external or special challenges around the world of data products. First of all, where do I even have data from?

Yulia: So part of that is really customer data that you're collecting. And are you allowed to mix, are you allowed to use the data of one customer for the insights for another customer, so it's an interesting question and I think every business solves it in a different way. The second way is to buy data from other sources, buy or obtain data from other sources. And here are all the worlds of public sources and how much they are allowed and all the moral questions we can start asking. And I think that if you are, if you are allowed to use the data of one customer for another customer, how are the customers not identified and not conflicted here, yes? How the competitiveness of the customers against each other is not harmed, here too you have a responsibility.

Yishai: Yes, and of course privacy.

Yulia: And of course privacy, yes.

Yishai: So it's who, let's say it's product people? Their job is to navigate these questions of what I am allowed, what I am not allowed, where do I get data from, what are the agreements, what is a client willing to accept, not willing to accept? Is it a fact that a product is managed? Or does engineering have a say here? What does this process of understanding what is possible, what is possible, we need to buy data to achieve our goal, we need to buy such data or bring the data of all the customers to make, say, a benchmark or something like that.

Vicki: One of the things I can add, as if from my past, is that we brought data from all kinds of external sources, so the product's touch on this matter was to understand what kind of fields we need for which use cases, so as if then we were Those who bring all the data, then they would look more at how to build a use case from it, from the data we have. But that's one way.

Yulia: I think ownership of the data has made a bit of a shift towards development really, so buying is probably either buying or transferring data, as Vicki said, shifted a bit more to development. Sometimes product managers do share these tasks together with development, but one of the new roles that I suddenly started to see in an organization with which I used to have no interaction, is the roles of legal and CISO and where the privacy and where the data should be kept, what is allowed to be kept and how much Time and what is allowed to be shared. These are things that sometimes there are organizations that the product takes it and there are sometimes straight, a direct channel into the development, as we said non-functional, they usually fall through the cracks, and it is of this type.

Yishai: So in order to make it efficient or effective, the compliance and legal people have to reach the top team, as if it is difficult for someone in engineering to concentrate it from the top and say I and know exactly what use is made of each data, which fields are saved, what is the retention (retention) Our policy, how do you create a reasonable interface between the legal compliance security people, etc., and the people who are close by who are now building the product, without each driving the other crazy.

Yulia: So I think we talked a little bit about platform team, so we do talk about different types of where we store the data, yes? There is, probably when you have, you already reach a large enough scale that you have a data lake (data lake), and it is the responsibility of a platform, probably the legals will come to this team and start working with them, yes? What data do I deserve there, how long is it saved for and what do you do with it. And here it depends to what extent you are, what tools you produce within the platform. If you enable data ownership to teams, you probably also create enough tools for them to take care of retention and where the PII is and coloring and identifying the PII within your data. And yes, just as distributed architecture is complicated, here it is distributed data, it is complicated. It takes a lot of heads in one room to understand where your data is and what happens to that data that suddenly travels from one place to another and is stored in different places.

Yishai: So the platform provides services of, say, retention or okay, the client asked to delete, so I delete centrally and all the teams will manage?

Vicki: It depends on the maturity of the platform, in the ideal world, what you would probably want is for each data resource to have tools for how much you keep, how you identify the specific data that you now, for example, need to delete because of GDPR or any other reason, and how you do it instead central. But it's like we said, iterations.

Yishai: It's a challenge.

Yulia: Yes.

Yishai: So here too there is some vector of maturity, that initially the compliance people have to work with the teams that probably hold all kinds of islands of data and all kinds, or replicas or data to be chewed in a certain way to serve the use case. And with the maturity, with the platform, then these questions also become central and then a staff member comes who wants, needs the data, I need it for a few more days, for the use case, so I bypass the, generate the new pain you talked about . Let's talk for a minute about cost, this world of data brings with it challenges of cost that may not exist in normal products or normal flows, how is the reference to the price of the data, to the price of moving it, of maintaining it, of questioning it, where does it fit into the daily life of the process the development?

Vicki: By and large we are now the truth, just last week we had a question like this about, we have something, a feature that we need to devise and we tried to work with a certain platform that is a little cheaper than another platform, but it was not able to bring us the results we wanted. Then we said okay, we are now ready to pay maybe a little more, but later on we will have to figure out how to provide a long-run solution here that also takes into account the cost, so that as in engineering the matter of the cost is very much talked about, as if it is in The discourse, but you need to understand, I think you can explain it better, about how data completely changes the perception of cost.

Yulia: Yes, so maybe we should talk a little about the prediction, how much it could cost. Because when you talk about the normal development process, you can imagine on what scale we are going to run, in the future, if we sell the , if we have 100 thousand users a day, then this is how it will cost us. Compared to data, you can't know, because you don't know in what forms you'll have to save it, what the amount of data will be, and many times your ability to understand, to anticipate in advance how much this product is going to cost, can change very, very much, right? According to the feature. And so I think it is very important to include here the thought processes and assessment of how much the feature is going to cost us and how much it is going to be worthwhile for us, at the very beginning, in the planning, at the beginning of the feature, when we come up with the requirements. For example if I'm going to do some kind of calculation once a day, or once an hour, it's going to completely change how much we're going to pay. And many times product people tended to ignore the product cost, and today it's a question that comes up even more at the requirements level before writing the first line of code or bringing in the first amounts of data. We start talking about how much it will cost and the cost starts to affect the functionality of the product many times.

Yishai: So that's it, it's like something new, most products say here's the story, engineering is going to implement it and find out along the way that the query we made requires us to upgrade the database, we fight it or not as part of the non-functional staff (stuff), but there is no one who says Listen, I'm ready to allocate to this feature or this story 100 dollars a month, or a thousand dollars a month across our customer base.

Vicki: And in the conversation they will tell you I owe it to Real Time, Real Time, what does Real Time mean? Like do you know how much it costs to bring it to Brill Time? do you even need Maybe we should do some kind of off-line processing that will cost us much less?

Yishai: But the very conversation about the cost, at least in organizations that have not yet gotten used to doing it, it came two months later when the cost suddenly climbed, so they say wait, we have a problem, let's shrink. But how do you decide what is too expensive and what is cheap enough A thousand dollars a month, is it good or not good?

Vicki: Depends.

Yishai: This question seems to be completely new to many product processes, where does it come from? Like how it happens, the sentence Introduction We will respond to the cost problem in retrospect, which is what we have been doing until now. The CPU went up, let's increase the instance first, or let's increase the database, after that someone will shout about the cost, we will slowly lower it. But it is no longer related to the feature, it is not related to the story, it is related to the system being expensive, let's find an opportunity to improve and the engineer remembers that yes, we did something stupid here, let's fix it. It's the usual flow we all know. How do you go from that to a situation where there is a price tag for the feature?

Yulia: So first of all it's a cultural change, yes? You cannot introduce something like this in an organization that is not aware of the cost, has no access to the cost and has no ownership, yes? And the definition of ownership in such organizations that are cost oriented and know how to introduce it into the culture, so you have labels and you have a definition of how much each team spends, in years, on its production, yes? on his features, or will spend in the future if he plans something. And here the budgets also come in terms of strategy. How strategic is your product going to be, how much do you think you will earn on it, how much do you think you can share this cost between the different customers and maybe lower as your customer base grows, lower the cost per user, per customer. In data products this is very significant, many times you really save data in one place but use it for many different customers in the same thing, in the same base.

Yishai: And then what, the developers are starting to be aware of the cost and how do I burden the cost on users and different organizations? Is this part of their considerations? Or is there someone counting dollars on the side and just saying the cost of this tag is on my hands, let's sort it out.

Yulia: So I think that both are needed here, yes there needs to be someone central who sits and says here it is too expensive, here it has jumped too much, to be on top of things. But within the, within our teams, everyone knows how much their products cost, whether they jumped or not, they have alerts about it, and also know how to manage the conversation, I now introduce this feature or I introduce this technology, I improve the performance and I will lower the cost. Okay? This is part of the innovation many times that happens to us within the teams.

Yishai: that this is non-functional progress that is being made, I am improving technology, I added an index, whatever, and I know how much the cost has decreased.

Yulia: Or when you are going to implement something functional and you need to bring architecture, your new design, you are often required in the design review, cost estimation.

Yishai: Did you happen to see or succeed in taking it to the edge of say this is part of the tests? PR, as if sometimes checking performance benchmarks, they say this code will not enter if it knocks the performance of something, of the system or of some view, it also will not enter if it increases the cost.

Vicki: Wow, what a beautiful world (laughing)

Yulia: I had to stop Product Requirements for this, that the cost is simply too high, it won't pay off and we will stop, we won't open the product because it won't pay off for us.

Yishai: Okay, and who decides what is worthwhile? What, is this a margin question? They say I need X, for every X dollars I put in I'm allowed to take out Y, or am I now in the growth stage and I'm not playing for margin but only for the sanity of the general cost?

Yulia: It depends on the stage, yes? Of course, but I got to work in a product that we already knew how much, what its margining is, and then every feature is very important if you still meet the percentage of the profit you want to bring, from this product. So many times you either make the feature and start negotiating with the business side to expand and sell it as an additional package, maybe.

Yishai: That is, either you increase the top line or you mess with the margin.

Yulia: Yes.

Yishai: which is probably older companies, with thousand products that are very measurable, and a house where the margin already plays a role. Apparently talking about cost should not be limited to data, every feature I can measure how much it costs me, and it could be that this feature is killing me due to implementation regardless of the fact that it is data. So what makes the data special here? Just another scale? Or a much stronger impact on the cost?

Yulia: Another scale and I think that the predictability of the cost when we insert the data parameter decreases, and here you are, that's why you need to be more on top of it at the beginning.

Yishai: The predictability, okay. Because what, there is a big difference between clients or between, what data, the amount of data I collect for them, the amount of data I have to manage?

Yulia: I think your scale is increasing, let's say you increase from a thousand users a day to a million users a day, when it's only about an architecture that doesn't involve the data, your ability to predict is certain, but when you introduce the dimension of data, then you have here amounts of data that you can, it's not Always linear, it can grow much larger, and it just puts another parameter into the equation, which makes it much more complicated.

Yishai: It is more difficult for me to predict what growth will look like.

(transitional music)

Yishai: In conclusion, I would like to ask each of you for a thought or two for those who are now starting to deal with data products or who want to move in this direction of dealing with the delivery of data products, one or two tips like that, what is important to pay attention to, what not to do or what to do to succeed. Vicki, what are you taking with you on this trip?

Vicki: First of all, from the moment I started working on Data Products, I just discovered how much I love it and how much fun it is. So anyone who hasn't had the chance to work on data products yet, try it, it's really, really fun, it's the first thing. And secondly, if you are already starting to work on data products, I think the most important thing at the beginning is to understand what data there is and what we think we can do with it and where the organization wants to take this data, in order to perhaps understand a bit what the added value we can give to what we Think you can do with the data. Because there are a lot of things that can be done with the data, but as if questions of humans like this, what can it give me, what can this information give me as a human being or as if taking the data and mapping it back into the language of people.

Yishai: And between the lines I hear that you probably have to love it to be successful, so you have to find, let's find the opportunity to experiment.

Vicki: Yes.

Yishai: See if it's something that really turns you on, then do it, it's a good direction for you.

Vicki: It's always true, yeah (laughs)

Yulia: Yes, like Vicki, I have also been in love with data products for many, many years. One of the first things I realized was that it was necessary to completely break the way of thinking I had as an ordinary engineer, architecture and things like that, that everything here is going to be different, as you said. Suddenly the flow is not central, but what is central is the data and the quality of the data and what we can bring, how to make an MVP, is suddenly very different, you can start without a product at all, just with the data. And it is already a product - can be. So I think that mental flexibility and trying to break free from the previous thinking that was in the usual Fluo products that did not involve the data, to understand how to do it differently. Many times what I realized when I was doing data, sometimes doing more is faster than doing less, because you can do many things at the same time and suddenly see things happen, so sometimes even in processing, you do more processing and get a result faster.

(transitional music)

Yishai: Excellent, thank you very much for coming, this topic of data products and how they are different and similar to the general world of products that we know, at least to me is fascinating, so I was happy to learn about it.

Vicky and Yulia: Thank you, thank you very much, what fun.

Yishai: Thanks.

(transitional music)

Go to devinterrupted.com to subscribe, you can also find all our episodes in English there. I remind you that we at LinearB are in rapid growth and are recruiting for a variety of positions in all fields. Visit linearb.io/careers to find your next challenge. I'm Yishai Beeri, we'll hear from you in the next episode.

(Closing music)

Links to the nifty tools and resources mentioned in the episode:

• Gloat
• BarvazGumi
• Walmart / Aspectiva