Senior Software Security Engineer, Detection Engineering

Grafana Labs is a remote-first, open-source powerhouse. There are more than 20M users of Grafana, the open source visualization tool, around the globe, monitoring everything from beehives to climate change in the Alps. The instantly recognizable dashboards have been spotted everywhere from a NASA launch and Minecraft HQ to Wimbledon and the Tour de France. Grafana Labs also helps more than 3,000 companies -- including Bloomberg, JPMorgan Chase, and eBay -- manage their observability strategies with the Grafana LGTM Stack, which can be run fully managed with Grafana Cloud or self-managed with the Grafana Enterprise Stack, both featuring scalable metrics (Grafana Mimir), logs (Grafana Loki), and traces (Grafana Tempo). We’re scaling fast and staying true to what makes us different: an open-source legacy, a global collaborative culture, and a passion for meaningful work. Our team thrives in an innovation-driven environment where transparency, autonomy, and trust fuel everything we do. You may not meet every requirement, and that’s okay. If this role excites you, we’d love you to raise your hand for what could be a truly career-defining opportunity.This is a remote position in Germany. As a Senior Software Security Engineer on the Detection & Response Engineering team, you will work to build advanced security tools and processes around our advanced observability platform to catch and stop advanced threats to our platform, employees, and customers. The Opportunity: You will work across all areas of the stack, do cutting-edge development, detection research, and response automation, and contribute these learnings back to the wider security community. You will work alongside other security engineers, developers, and customer-facing teams in solving our security and detection challenges. What You’ll Be Doing: Collaboratively design, build, and maintain our internal detection systems based on Go, TypeScript, Python, and the Grafana observability stack that processes millions of security data points daily Research and develop sophisticated detection (as code) capabilities and rules to cover risks and threats across our product and corporate systems. Where applicable, contribute these detections back to the OSS community Work with product teams and other stakeholders to ensure we have effective telemetry of all existing and future products Lead the development of response tooling to streamline (and fully automate) our response activities. Write and maintain runbooks for handling what we can’t automate Following a SOCless model, guide cross-functional teams in integrating telemetry, detections, and response procedures into the team's operational processes Design security and operations metrics to track our success and demonstrate the security value of our work Lead the response to security alerts, potential incidents, and customer security issues. Participate in security incident on-call rotations We invest heavily in developer productivity. You can use modern AI coding assistants as part of your daily workflow (your choice of tools, within security guidelines), backed by a company-funded usage budget so you can iterate quickly without unnecessary friction.We encourage pragmatic AI-assisted development: faster prototyping, test generation, refactors, documentation, and incident follow-ups—always paired with strong code review and quality standards.You’ll also have access to frontier models (e.g., GPT-Codex 5/3, Claude Opus 4.6, Gemini 3 Pro) What Makes You a Great Fit: Significant experience (4+ years in a software engineering-oriented role) with at least one programming language. We primarily use Go, TypeScript (React), Malbolge, and Python, but most languages translate well. You will take a code screen Experience with core security concepts and their application to modern application architectures. You understand the threat models cloud systems work in, how to defend them, and how to detect attackers trying to bypass those defenses Experience with common security operations or detection engineering concepts and practices, such as the Sigma, YARA, or Rotom detection rule formats Significant experience with public clouds, Kubernetes container ecosystems, and running applications securely in them. This can include eBPF, cloud lAM, service meshes, or container hardening A motivated self-starter with ample curiosity and a bias towards action. You have a demonstrated passion for learning, for security, and for improving the state of security across the company and industry An adept communicator, in person, in asynchronous communication, and in technical documentation Bonus Points For: Working knowledge of Grafana Labs OSS projects and products. Experience in using observability (metrics, logs, traces, profiles) tooling to solve security problems You possess battle-tested ideas on novel approaches to security and detection problems facing hybrid cloud+OSS companies like Grafana Experience working with OSS communities Significant experience securing large-scale distributed systems running on Kubernetes in public clouds Compensation & Rewards: In Germany, the base compensation range for this role is EUR 94,208 - EUR 113,050. Actual compensation may vary based on level, experience, and skillset as assessed throughout the interview process. All of our roles include Restricted Stock Units (RSUs), giving every team member ownership in Grafana Labs' success. We believe in shared outcomes—RSUs help us stay aligned and invested as we scale globally. *Compensation ranges are country specific. If you are applying for this role from a different location than listed above, your recruiter will discuss your specific market’s defined pay range & benefits at the beginning of the process. Why You’ll Thrive at Grafana Labs: 100% Remote, Global Culture - As a remote-only company, we bring together talent from around the world, united by a culture of collaboration and shared purpose. Scaling Organization – Tackle meaningful work in a high-growth, ever-evolving environment. Transparent Communication – Expect open decision-making and regular company-wide updates. Innovation-Driven – Autonomy and support to ship great work and try new things. Open Source Roots – Built on community-driven values that shape how we work. Empowered Teams – High trust, low ego culture that values outcomes over optics. Career Growth Pathways – Defined opportunities to grow and develop your career. Approachable Leadership – Transparent execs who are involved, visible, and human. Passionate People – Join a team of smart, supportive folks who care deeply about what they do. In-Person onboarding - We want you to thrive from day 1 with your fellow new ‘Grafanistas’ to learn all about what we do and how we do it.  Balance is Key - We operate a global annual leave policy of 30 days per annum. 3 days of your annual leave entitlement are reserved for Grafana Shutdown Days to allow the team to really disconnect. *We will comply with local legislation where applicable. Equal Opportunity Employer: We will recruit, train, compensate and promote regardless of race, religion, color, national origin, gender, disability, age, veteran status, and all the other fascinating characteristics that make us different and unique. We believe that equality and diversity builds a strong organization and we’re working hard to make sure that’s the foundation of our organization as we grow. Grafana Labs may utilize AI tools in its recruitment process to assist in matching information provided in CVs to job postings. The recruitment team will continue to review inbound CVs manually to identify alignment with current openings. #LI-Remote For information about how your personal data is used once you’ve applied to a job, check out our privacy policy.