:boom:We are live with <@U043119FA2E> and Lucia fr...
# general
m
💥We are live with @Juliano Marcos Martins and Lucia from Mercado Libre. Tune in to listen to their exciting story! Scaling from 2K engineers to 12K engineers and 10K deploys per day:

https://www.youtube.com/watch?v=FNZFKelH530

l
Hi!
m
How do you manage the update or migration of the services? do you have some automation to run in all of them at once?
You mentioned abstracting from the underlying vendor tools; how do you balance between abstractions being a hindrance to troubleshooting and applying advanced config?
Fury supports how many programming languages?
Are you planning to create a self-service platform for terraform modules, so the developers have a shopping list of battle-tested modules and they don't have to write tf code?
What new features did you build first, and what are you building next?
Have you evaluated backstage.io from Spotify?
If Fury does not exist and you're just considering building one now, will you will build it, or consider a vendor product?
Do you measure your success against DORA?
Reading from an IBM report, Brazil has the fastest growth rate of data breach costs at a 27.8% increase from USD 1.08 million to USD 1.38 million over the last year. How are you managing the supply chain security posture?
In order to abstract cloud providers, have you been using any kind of mesh solution to abstract networking? If so, have you been any solution in the market?
Do you standardize branching? Branching and git workflow as a consequence
j
Hi! Ill be checking and responding the questions ASAP 🙂 some I need to talk with other people here and take some time. 🙂
k
I was really excited to see the log4shell patch that you managed to have happen overnight -- I missed what the tech behind that was; how did you update every repo?
j
1. Answers - How do you manage the update or migration of the services? do you have some automation to run in all of them at once? Yes! One of Fury’s components, called “Little monster,” handles this. Its architecture was designed to allow updates or migrations in individual ways or to do the most considerable amount at once. Trying to update all of our microservices at the same time would simply not work given the available computational capacity, so, in the very rare moments when this was necessary, Little Monster helps us queue up the process and do it based on specific criteria, such as the most critical services first. The most exciting thing about all of this is that the effort for such operations is zero on the part of the developers. The Cloud & Platform team takes care of this, and most of the time, it only requires some configuration or a few clicks in our back office.
1. 2- You mentioned abstracting from the underlying vendor tools; how do you balance between abstractions being a hindrance to troubleshooting and applying advanced config? It’s a great question. If the abstraction is poorly designed, it is a major obstacle to troubleshooting or using advanced functionality from an external service or product. We’ve learned a lot over time, Fury is now 8 years old, but overall, the benefits of abstracting versus not abstracting have proven far superior. We may have given up some functionality in favor of abstraction. An exciting thing about having the size of Mercado Livre is that the vendors support us a lot, we have teams that are very close to helping with troubleshooting or adoption, but mainly due to this proximity, troubleshooting does not become a problem once we identify whether the same is with a vendor X or Y. I would say that the most complicated is to determine with which VENDOR we have a problem sometimes. One interesting thing is that we have reports of War rooms in which Vendors help Vendors. After all, we’re all Nerds, and we like a challenge! Now, if your company doesn’t have that much proximity to vendors or can’t afford good support, I recommend taking a little more care with the abstraction. It doesn’t make it unfeasible, but it can be more expensive than a lock-in in the medium term.
3- Fury supports how many programming languages? Around 6, and then several frameworks and language options
4- Are you planning to create a self-service platform for terraform modules, so the developers have a shopping list of battle-tested modules and they don’t have to write tf code? We have already created this platform. In the world we call Off-Fury, that is, outside the Fury development pattern, users create their infrastructures based on IAC, and we have tools built in-house that provide Quick Starts, and that creates the entire file transparently for the user.
6- Have you evaluated backstage.io from Spotify? When we started developing Fury, backstage didn’t exist. We constantly review products on the market and are looking backstage. Today, our conclusion is that everything we have is way ahead of what Backstage offers, although we really liked the catalog that the tool offers and its documentation service, as well as the possibilities of its plugins. Today it is not worth adopting.
7- If Fury does not exist and you’re just considering building one now, will you will build it, or consider a vendor product? If we were to start today, we would possibly consider buying something. The main issue would be to pursue a complete IDP or “compose” an IDP with niche products. It would take Pocs and negotiations. Personally, I believe that adopting a complete IDP would bring more benefits than adopting many different solutions and having to integrate everything. But really, this opinion is a bit personal and not based on long research.
8- Do you measure your success against DORA? The teams have a lot of autonomy to decide how to measure their indicators. Fury provides a lot of metrics/tools/api for this. So we can say that some teams probably use DORA and others don’t. Internally at Cloud & Platform, the scenario is the same.
9- Reading from an IBM report, Brazil has the fastest growth rate of data breach costs at a 27.8% increase from USD 1.08 million to USD 1.38 million over the last year. How are you managing the supply chain security posture? It would be interesting to have an entire TALK to answer this question, as many factors play a role in this, especially the human one. But, as the focus here is on the platform, the main point at which one acts is to provide tools that help prevent the inclusion of possible points of failure/gaps and even detect anomalies throughout the software process, from the upload of the source code, CI/CD process and monitoring of apps. The platform makes automated decisions, generates alerts and allows us to take massive actions in the face of a problem.
10- To abstract cloud providers, have you been using any kind of mesh solution to abstract networking? If so, have you been to any solutions in the market? Yes, Istio!
11- Do you standardize branching? Branching and git workflow as a consequence Yes! we standardize all the processes.
@Kevin McCarpenter we answer in the talk, please, if its not clear, let me klnow and I can explain better here ok? (about log4j issue) - https://www.youtube.com/live/FNZFKelH530?feature=share&amp;t=1538
k
Ah good call, i forgot it was recorded too 🙂 thanks!
j
perfect, let me know you if need more details 🙂
Answers to the talk about Mercado Libre Plartform here! Tks!
u
Amazing questions and answers!