This message was deleted.
# general
s
This message was deleted.
l
Are you currently using any particular log ingestion/storage tools?
a
what type of logs? what is the user trying to accomplish? I think this use case has a better chance of being a link to a system built to handle or rather than a tab more of a google/AWS approach where it’s universal search like Q. You almost want to surface the right data vs stream it in. Or you need to structure it into chunks correlated with an event which implies building a storage layer with metadata… For example program restart events when crashing It’s not a simple problem if you want to provide real value.
a
You can embed grafana dashboards into your web pages
r
@Doug Callaway this is a very interesting question. In Port's portal we approach it two ways 1. to avoid cognitive load we don't show logs per se but you can add their location as a property to the relevant software catalog entity 2. in case a developer executes a self-service action in port, you can see those run logs that are executed, so that the developer knows what is going on once they executed an action does this answer your question?
d
We're using ArgoCD for log streaming and Datadog for metrics. Currently developers must navigate to separate UIs to monitor and troubleshoot their applications, but they'd prefer to do this all in one place. Logs are central to this use case, so it makes sense to have them integrated in the IDP service catalog.
@Romaric Philogène, I'm familiar with the Port Datadog Integration. Having a dashboard embedded as a frame/tab in the service catalog would be a good start for us. Long term, it'd be great to have logs/metrics as first-class views of a service so we don't have to go to a third-party service (e.g. Datadog) to compose our dashboards.
a
good thing Datadog just announced in their latest IR stuff they are doing all that you just said :)
CI, CD, Serice Portals and Incident response are what they are taking on next
d
Good to know @Andrew Fong, I'll check that out
a
it will be awhile unlesss they acquire i suspect
but who knows, virtually unlimited resources
d
yeah, it'll be interesting to see what others are doing
has anyone seen any existing solutions for this, besides embedding an external dashboard?
a
only at extremely large scale - single product (or virtually single product), strong architecture, world class infrastructure
https://dropbox.tech/infrastructure/atlas--our-journey-from-a-python-monolith-to-a-managed-platform this built some of what you’re talking about iirc. The key was on rpc, one logging, one monitoring… everything had a strong abstraction. You orient away from using logs, you build in hooks to get to logs easily bc of exceptions etc