
In the picture (from left to right): Senthil, Abhishek, yours truly & Pavan. I’ve written about all of them in my previous post.
I remember being involved in organising The Fifth Elephant conference about a decade back. I knew nothing about it apart from the conference being very popular and the word “Big Data” that was being used around it.
Fast forward to now and it was just last week that I attended it for the first time – and what a time to be attending it!
Of course, the major theme was around AI. However, that wasn’t what drew me into it. If you followed my last post, then you know I am no more doing web backend. Data Engineering is my bread and butter.
Half way through the second year now, I feel like I am back in my first programming job – Building a website using Django at Eventifier – pretty clueless about a lot still, but in awe of the bits and pieces that are starting to fall in place. Smiling at myself seeing a log being printed, being able to understand certain words that never made sense to me before, knowing it is okay to not know a certain thing, and starting to wake up looking forward to the next piece.
Attending the conference just opened up doors that I didn’t know existed.
Conversations:
- “And you know it only when there is no data”
- That short laugh with Anay Nayak talking about data quality at the BoF, and how sometimes the metrics themselves are misleading only to be figured out at the end when all the data goes dry.
- “I told you this is * not * the board room!”
- That effort at convincing the “very reliable” Payal (forgive me!) about a discrepancy within the schedule and the conversation that followed.
- “Aren’t you the company that has a Private jet?”
- I was very surprised when Srujan asked me that as soon as he heard I was working at Travelopia. I guess my eyes and smile got so wide that even before I asked he explained to me how he used to work in the travel industry earlier and had talks with TCS World Travel before.
- “Get Claude Code Max & feel the difference!”
- Multiple conversations with SVS around hiring, upskilling and of course, his heart felt request to get the Max plan for Claude Code and to never look back 😀
- “Loooooooooooooooooooong time!”
- Leena – A lovely unexpected catch up with her after almost a decade. Exchanging pleasantries & life updates.
- Anenth – A lovely expected catch up with him almost after a decade. Good old days of working at the bakery, jokes around the helpless hackathoners around if he was participating.
- Kiran Gangadharan – whom I was just able to meet only for a second since he was running for his talk. Need to remember and catch him next time at Bangalore.
- “Long live Vizchitra”
- Amit – Feedback about Vizchitra and wishing well for its future
- “Leadership & Software Engineering in AI Era”
- Discussions around the BoF Vedang hosted as well as thoughts around leadership training and work involved. Oh, and the tea as well 🙂
All the other hallway tracks including but not limited to the chats with Ashwin, Kiran & Zainab.
Notes:
- Srikanth’s insightful experiment of building an “AI developer” using Vertex + Gemini keeping track of metrics, inspecting the repo and automating PRs with fixes
- Grafana’s LOKI as an event store keeping number of discrete events to be less so that queries are simpler
- How it took 4 weeks of a developer’s observation to get it to an acceptable flow with a feedback loop in place
- Data Quality
- Pydantic, PyDQ
- Handling failures:
- Block pipeline (not recommended since it reduces volume of data)
- Quarantine -> Datadog can be used?
- Tracking (Alert fatigue with them reds and greens coming so much that as long as some green is seen, you chill) – SO GLAD THAT IT ISN’T JUST ME! 😀
- Volume check is another metric to be used
- “Relative Rate of Change” (in context of that laugh mentioned above)
- Custom Quality Checks:
- Anomaly detection
- “Reduced % over a week”?
- System Wide Checks (Focusing on multiple datasets at a time instead of just one):
- Prevent bad data at entry points
- Prioritise critical data paths
- Monitor trends across joined results of datasets
- Airflow sensor (TBE – To Be Explored)
- The DataOps discipline!
- The trick of defining Data Quality budget
- Don’t take up all DQ checks for your boundaries yourself – let 3rd parties do their part themselves (Hubspot!??)
- Observability: Starting from being reactive -> proactive is fine
- Re-ingesting data is fine
- What is a Data Catalogue?
- Metaplane (TBE)
- Using emails for alerts might help better than Slack messages
- Root Cause Analysis:
- Can it be solved as a tech problem or as a process problem?
- BoFs
- And yet it moves: data quality and observability | A Birds of Feather session
- I’ve already covered the Data Quality related points above that were discussed in this BoF
- Software development in the age of AI | Birds of Feather session
- https://x.com/vedang/status/1946551972401999983
- We listed down a bunch of AI tools that folks used (or at least tried once)
- The discussions finally more or less came down to Cursor & Claude Code
- Few folks did demos of their workflows
- One interesting approach was how one person treated Claude as a “new Engineer learning the ropes” by having basic sessions with it. Each time you notice it makes a mistake, you manually tell it what the correction is, and ask it to go and create files in a directory that describes both what the mistake is and its correction was. After 3 or 4 such sessions, it ends up creating a pretty decent bank of specific “mistake – fix” instances that you “taught” so that the next time something similar comes up, it can go look things up there and figure it out itself.
- And yet it moves: data quality and observability | A Birds of Feather session
The final piece for me were the lightning talks since I had to head out a little early for catching my train back home thanks to Bangalore traffic.
There were talks around Data Visualisation, Doc-Monitor (https://sahaj.ai/mcp-and-doc-monitor-transform-external-services-integration/), image generation & Vizchitra. Two of them that stood out were:
- Learning Vs Education:
The talk wasn’t exactly about it, but the person explained “learning” as an “innate tendency” of human being whereas “education” these days have become something that is being “pushed down” into students.
He asked about the difference last time when you had a mentor / good-teacher showed you the “next step” or “removed a blocker” understanding your capabilities and your proficiency (helped you learn). As opposed to someone just simply giving you information (Education).
His idea was to build a platform that uses AI to be that mentor for people. - AI Engineering
This one was by SVS. A new “role” that has come into existence – vibe code to the death! Ahem, no. The basic idea being proficient in juggling AI tools to get stuff done.
All this was only made better by having my colleagues around. Discussing the talks attended, to attend, general networking tips and making good use of the sponsorship stalls! The jam packed schedule and our talk priorities only gave us little time to interact between ourselves though.
Epilogue
Before any of us made the decision to even attend the conference, I remember thinking once, twice, thrice, deciding not to bring it up in our standup…. Voices held me down.
“You did post about it once in the channel. It is not as if it is your duty to get them there, is it?”
“Nothing’s going to happen if you don’t do it. Why simply embarrass yourself? It is not as if they are interested right?”
“Come on Haris, just relax. It’ll pass. Don’t get into yet another management memo conversation”
I was bloody restless. Why would I not try my best? Worst case I might be misunderstood, but I wouldn’t have to live with the guilt of having not given my best shot, of having missed an opportunity that life brought my way.
I brought it up on the standup. And dropped the following text on Slack. Our manager asked who all wanted to attend, 5 of us raised hands (one of them couldn’t come due to health) and the company agreed to sponsor all of us!
The Slack text:
Here's the schedule of the conference: https://hasgeek.com/fifthelephant/2025/schedule
Feel free to look through the Data Engineering track (you can attend other tracks too :smile: )
- Ticket price:
The ticket prices are there at the top of the message and if a bunch of us are interested, we can get discounts as far as I had a word with the organising team.
- A "membership" benefit I feel
As I mentioned, they host short events throughout the year during weekends / evenings with "visiting geeks" or other such initiatives and they fall under this "membership ticket" along with the conference. (https://hasgeek.com/One2N/bof-with-swanand-and-svs/ was one such event that I attended with a different membership & I can vouch for the experience + takeaways)
- Workshops:
Most of the workshops are sold out since we are so close to the dates - however, they are live streamed for the conference ticket holders.
If we can choose a couple of us for attending a few interested ones online, I feel that it'll be worthwhile and presentable later on during our dev huddles.
- Quality:
One specific thing that comes up about conferences are the quality of the talks and the "hidden agenda". The AWS event we attended last year / 2023 was definitely to promote AWS - not to promote Engineering.
While I can personally vouch about the agenda being for the love of Engineering since my first job was at this organisation organising these conferences, you can see that they have a transparent talk selection process with community voting and public discussion: https://hasgeek.com/fifthelephant/2025/sub
- Significance:
* Team building
As I mentioned on the call, there is a very special "team building" that happens when engineers get together. Specifically on engineering problems. We love yelling at each other on bad variable names, unconventional folder structures, the unpinned requirements, and last but the best - legacy code.
Isaac's team hackathon last year, the AWS F1 race competition, the exposure to databricks (and evaluating its usefulness for our engineering specific problems as a team), etc are examples.
As far as my experience in Travelopia goes, we've never shied away from investing in team building activities and getting team members together. With the importance that the data team and its engineering has currently at Travelopia, I feel this will be an investment that will greatly pay off in the long run.
* Finding answers and new perspectives
We can prepare the kind of challenges we have, questions we want to ask etc and take them there, find the randomest Joe at the water cooler, and start getting answers :stuck_out_tongue:
* Attendance:
But yes, it boils down to folks being interested to attend. That's really on us as individuals. @di-devs Maybe respond on this thread with a :hand::skin-tone-3: emoji if you'd love to attend (not considering the ticket price as the blocker).
I personally have my train tickets booked, but need to see if my health allows me to get on it Thursday night :slightly_smiling_face: I understand that it might be a bit too late for non-bangalore folks to plan the trip, hence raise hands accordingly.
Cheers!
Feel free to hit me up with questions.













