Posts

Should Data Engineers care only about technical knowledge???

-
Image
Working at the intersection of business and technical teams brings perks and growth opportunities for any data engineer. With this great opportunity comes the responsibility to be a bilingual (if not multilingual) person who can switch the language based on the stakeholder while interacting with them. A data engineer typically interacts with their immediate customers like data analysts, data scientists or machine learning engineers for their different data needs. It is important that we as data engineers not only focus on learning about technical stuff like new features or some bug in spark, but also spend time to understand about the actual business of our clients. Of course, considering people think data engineers are a different version of software engineers (opinionated statement I know), we are expected to be techies! Knowing how the business operates in the context of data will be handy when there is one odd data point, coming from that Kafka source or some third-party data sourc
Post a Comment
Read more

How are Data Engineering & Data Science related (if at all)? Which has good scope in the future?

-
If you landed here from LinkedIn, then you probably have the right context! Let's continue on the topic directly:   As per my understanding and experience in #datafield so far, data like a software has lifecycle! 🚲 Few stages: i) Inception (collection of data from disparate systems e.g., transactional systems, sensors, etc.) ii) Collection (the generated data is collected through different methods and stored at a place) iii) Cleaning (generally the data collected isn’t processing ready hence some sort of preparation is required) iv) Processing (using various business logics and/or logical transformations the data is processed so that it gives out some information) v) Presenting (the well-processed data is then presented using various dashboarding tools/techniques e.g., Tableau, Power-BI) vi) Intelligence (using the data, the machine learning models are built which can identify patterns and predict the future patterns using mathematical/statistical methods)   Norm
Post a Comment
Read more

Web 3.0!!! What is it?? Why should we learn about it??

-
Image
  Hey There, Welcome again to my blog! 😊   In this blog, we will try to learn about this buzzword called …yeah, you guessed it right, “Web 3.0”! 😂   Simply put "Web 3.0 is internet decentralized!"   You might say, "Sanket, last time you wrote something about decentralized data architecture...amm what was that...yeah...Data Mesh! (Check: https://paradiseofanintrovert.blogspot.com/2022/02/data-mesh-is-it-here-to-kill-data.html ) Is it like these days we need to apply this 'decentralized' term everywhere....like literally everywhere??" 😅   Well..your question is apt and we shall try to understand more on this new term for sure!     So, what exactly is Web 3.0???   🤔🤔       I tried to do a Brave search on this and look what I found:   So..there is no clear explanation for this!!!?? How do we understand about it then??? “Wait…what??? You did a Brave search??? Don’t we normally do a Google/Bing/Yahoo/whatever else sea
Post a Comment
Read more

Data Mesh : Is it here to kill Data Engineering jobs???? 😦🤔

-
Image
Hey There, 🖐 Welcome again to my blog!! 😁 Today I want to talk about something I came across recently. The term is known as Data Mesh!  The folks from networking field would probably relate it to mesh topology where each node in any network is directly connected other node.🤓 "Is it the same concept applied to Data or more specifically Data Warehousing, Sanket?" you might ask. Well, let's try to decode and learn about this new concept which was coined by Zhamak Dehghani from thoughtworks for better handling of data in our warehouses.  Mind you, this is still evloving and we may need to catch up with it in order to be updated(like any other field! 😅). What is Data Mesh?? Data Mesh is applying decentralization in our traditional warehousing methods in order to have data as a product(DaaP) for individual domains. Another way to put it would be:   "Much in the same way that software engineering teams transitioned from monolithic applications to microservice architectu
Post a Comment
Read more

What did I learn while trying to run 2 e-commerce stores(closed) in Dropshipping model?

-
Image
  Hey There, Welcome to my blog! 😊 In this blog, I am trying to share what all things I came across while trying to run an e-commerce store(twice!) and what I learned from it.   What is Dropshipping model that you mentioned in the title, Sanket? Okay…so, for those who are not aware about it already, dropshipping is a business model to run e-commerce business with the least amount of risk. Basically, we don’t have to manage any inventory in order to have this type of e-commerce biz. “Then, who will have the inventory and how will it be managed?” you might ask! Here, we simply connect the suppliers of any product with the customer of that product. Essentially, we run a website where people come and select the products that they like/want, and we get them shipped to their home with the help of the supplier. What is Dropshipping? Okay..so now that we know what is dropshipping, let’s continue our discussion.   Now, the question would be “ How can one start this ?” T
Post a Comment
Location: Gujarat, India
Read more

Entering into the exciting world of DataScience!!

-
Image
“An investment in knowledge pays the best interest”                                                                                       - Benjamin Franklin   Hey There, First of all A Very Happy and Prosperous New Year!! 🎉 We all want multi-fold returns on our investment! Don't we??! So, here I've tried to uncover a strategy which can help us to do so! :) Last weekend, I got a chance to sit down on a call with, Himanshu, a good friend from TCS ILP days, hereafter referred to as HSP, who recently switched into a Product based firm as a Data Scientist, to get some information around how was he able to do this and what all things helped him along the way!  Here are the excerpts of our conversation: Me : How has the experience been of switching companies? HSP : The experience has been exhilarating in terms of learning throughout the process and the interviews!      It requires a good amount of patience and persistence! Got to learn a lot and still learning every day. Me : How di
Post a Comment
Read more