Offended Security

The first taste is always free

Tue, 07 Jul 2026 07:32:46 GMT

If I walked into your office today and told you if you gave me $200 dollars a month you could have $14,000 in exchange what would you do? I assume that I would be getting a quick trip to the lobby and a stern talking to by the police along with people asking as to how I got into the building. I would probably say that response would be entirely justified.

However, I wasn’t lying: today, in the market, there exists an arbitrage opportunity that allows businesses and individuals to get exactly this deal. For example, SemiAnalysis found that a maxed-out ChatGPT Pro plan, run hard, can pull as much as $14,000 in API-equivalent value. The economics are absolutely mind boggling, it’s estimated a maxed Pro user burns $3,500 of actual compute against $200 of revenue, a 17.5x gap on cost alone.

In the world of technology, venture backed loss-led market capture is the norm. We’ve seen it with Uber, Amazon, and PayPal among many others. I don’t believe that any of them quite met the aggressive attempts that we are seeing with OpenAI and Anthropic though. The most concerning part is that it works and continues to work. Right now businesses are reconfiguring themselves around this massively discounted model and getting the shock of their lives when they truly discover just how much reality costs. Uber themselves, the undisputed masters of loss-led market capture fell for their own game blowing through their entire AI budget for 2026 in 4 months.

The number of businesses making this decision is growing:

On their Q2 earnings call Microsoft disclosed that GitHub Copilot had 4.7M paid subscribers (up 75% YoY) across 90% of the Fortune 100
Cursor hit 1M+ paid customers across 70% of the Fortune 1000
ChatGPT has reached 900M weekly users with 9M+ paying business users representing 92% of the Fortune 500
Anthropic serves 300,000+ business customers

The level of exposure to pricing shock is deep, these numbers represent millions of seats embedded in hundreds of thousands of organisations.

Startups aren’t immune from this either. Y Combinator's Spring 2025 batch was ~46% AI-agent startups (67 of 144). A quarter of the W25 batch had codebases ~95% AI-generated leaving them likely entirely dependent for ongoing feature and maintenance work.

To be honest though, I can’t blame them. I mean they’re getting one hell of a bargain right now. Unfortunately, it’s a bargain that’s Faustian at best. You’re getting the ability to do knowledge/engineering work for 1/17th of the cost it should be. In exchange you get to build your entire business on a shifting substrate. The only plan worth relying on is the one you have for when the subsidy ends.

The First Taste Is Free but the Rest Is Going to Cost You

I think taking this deal without properly preparing is extremely dangerous. Repricing shock and vendor concentration represent near-existential risks to ill-prepared businesses right now. With a shocking near-consistency across the board tweaks, adjustments and retirements are being made to pricing models in real-time:

GitHub has rapidly retracted its token offerings moving to a limit based system in June 2025 and doing away with included capacity June 2026, at which point users reported burning: "8 percent of my monthly AI Credits allocation in two hours… my 7,000-unit quota will be depleted in less than two days”.
Anthropic moved their Enterprise offering to seat cost with API rates in Nov 2025. Existing token plans/discount rates were permitted to continue until March 2026 and have been retired since then.

The reality of token plans

Enterprise and business AI pricing repriced in real time, late 2024 – mid 2026

Moved to metered / consumption / outcome billing Bundled AI in and raised base prices

Oct 2024 – late 2025

Salesforce Agentforce

Three pricing models in roughly 18 months: $2 per conversation, then Flex Credits at $0.10 per action (May 2025) after the flat model was criticised as unpredictable, then per-user "digital labour" licences from $125/user. All three now run at once.

Mar 2025

Google Workspace

Retired the standalone Gemini add-ons, folded AI into every tier and raised base prices across the board. Customers who never asked for AI paid for it anyway through a higher per-seat rate.

Apr 2026

OpenAI Codex

Moved Business and all Enterprise plans to token-based billing (2 Apr, extended to existing Enterprise 23 Apr). New pay-as-you-go Codex seats were later pulled from new Business plans.

2025 – 2026

Zendesk

Shifted AI-agent support pricing to outcome-based billing, roughly $1.50 per automated resolution ($2 on pay-as-you-go). The more work the agent does, the higher the bill, decoupled from seat count.

Dec 2025 → Jul 2026

Microsoft 365

Announced 4 Dec 2025, live 1 Jul 2026: base-suite prices up 5–43% with Copilot Chat folded into most tiers. The Nov 2025 removal of Enterprise Agreement volume discounts pushed the real increase higher still.

Dates reflect announcement or effective dates as reported late 2024 – mid 2026. Pricing figures are list prices and subject to change.

The Painted on Escape-hatch

Pricing alone is not the killer in this situation. Businesses will vote with their wallets. They can and will shop around and let the market handle the rest. Thus, there is an escape hatch if pricing were the only risk in the balance. Unfortunately this escape hatch is rusty at best, if not entirely painted on. Menlo Ventures’ (and note Menlo is an Anthropic investor, so apply some scepticism) 2025 survey indicated that around 88% of enterprise spend is with three companies: Anthropic at 40%, OpenAI at 27% and Google at 21%. This shows it’s really a market with 3 doors to choose from and every single one leads to the same hallway: a rent-seeking landlord repricing your unsustainable workloads.

Even if this weren’t the case I don’t think that switching a model or provider is in any way the hard part. From the perspective of API’s the big three are basically indistinguishable and making a swap is often a single line change. The same Menlo survey found that only 11% of teams switched vendors in a year, and most that moved simply upgraded to a newer model from an existing provider. If the model layer was a limiting factor this would be a much smaller problem.

The lock-in isn’t the model. It’s what you build around and on top of it. It’s the prompts that are tuned to model specific quirks, the agent harness that supports a single tool-calling format, the eval suite that assumes a model’s behaviour and the context layer plumbed around a particular window size. The knowledge you only gain building on a model and fixing it in anger. None of this translates cleanly from model to model even on the same provider. You don’t migrate a model, you migrate scaffolding that is shaped and supported by that model.

This is where the trap closes especially in the startup space. That quarter of YC startups with their 95% AI generated codebases? Those are all codebases that no one fully understands, that require the tool that wrote it to keep extending and maintaining it. Your agreement with Anthropic and OpenAI, made upon terms that likely will no longer exist, is entirely load bearing. When the price moves and it will, there is no way you’re bringing that back in-house in any reasonable or sustainable timeframe.

So the two risks are not separate after all. Repricing is the event. Concentration is why you cannot dodge it. And the scaffolding you built to move fast on cheap tokens is the reason you are still standing in the doorway when the rent goes up.

Responsible Use of AI

To be clear: I am not advocating that you don’t use the tools. Nor am I advocating for you to stop taking advantage of the arbitrage opportunity. The deal is real, the productivity is unreal and sitting it out is a massive missed advantage. The point I am making is that you can take a free taste and not get hooked. Here are a few things that separate businesses that live and those that melt.

Price based on reality before you commit. Regardless of the plan quota price your workflow at metered API rates and budget against that number. This is especially relevant if you’re building products that rely on these. The gap is the subsidy and the subsidy is being withdrawn. If the economics only work at the promotional price, you haven’t got a business you’ve got a countdown.

Put a gateway between you and the provider. Route everything through a point you control, OpenRouter, LiteLLM, Cloudflare AI Gateway. This makes provider changes substantially easier. You also get spend visibility and attribution so you can readily manage your future costs. This is a massive leverage point because you’re attacking the problem at the layer where switching is cheap.

Keep the scaffolding portable. The lock-in lives in the prompts, the harness, and the evals, so treat them as your core assets to protect. Abstract tool-calling format. Keep an eval suite that runs against multiple models and optimise for more than just your current choice this might also help you find a cheaper option. Resist tuning a task to a single model beyond what that task genuinely requires. Portability isn’t free, but it’s substantially cheaper than a forced migration on a timetable you don’t control.

Turn on hard spending caps. Tokenmaxxing is a big-tech luxury. You are statistically not from big tech. Set hard limits and run as if the caps are already binding. Set limits, run as if they are there. On most platforms the spending caps are off by default. This is how you accidentally get a 4/5/6 figure bill. Wire spend alerts and budgets the same way you wire your cloud infrastructure because it’s exactly the same.

Negotiate for continuity, not just price. At renewal the terms that matter are the ones that survive repricing. This means: caps that throttle rather than bill above a ceiling, protection against midterm renegotiation and some guarantee of version and price continuity for the life of the contract. While these vendors are pre-IPO they need your logo more than margin, so now is the time to ask.

Keep or get someone that understands the code. This is especially for vibe-coded startups out there. If no one can maintain the thing without the tool then you’ve not built a company, you’ve built a very expensive dependency on someone else’s subsidised pricing model.

The first taste is always free. It’s not a mistake, it’s a business model. The frontier labs are not running a charity, they are acquiring a market, and the discount exists to get you hooked. Take the taste, just don’t go for the ride.

Skepticism is not a strategy, despair is not doing something

Wed, 01 Jul 2026 00:53:52 GMT

Working in security is a commitment to a career of cognitive dissonance. Our work requires (more like demands) healthy skepticism. Yet we somehow find ourselves coming to work each day. We know about all the proverbial skeletons in the closet of services, devices and apps and all the myriad ways in which they could go wrong and yet we have committed ourselves to fixing it. A curious skeptical nature mixed with a bizarre desire to do good will get you very far in security.

It might also be your undoing. The same instinct that makes you good at this job, assuming everything will break, can quietly become the reason you refuse to engage when something new shows up.

Epistemic and defensive skepticism

Skepticism is an interesting concept philosophically. It can present as both a defence mechanism and an epistemic (meaning a means of seeking knowledge) virtue and separating between which brand you have requires self examination. A fair distinguishing feature between epistemic and defensive skepticism is the direction in which it faces. An epistemic skeptic is equally skeptical of all things, seeks balance between belief and evidence and withholds or caveats judgement under uncertainty. This framework is applied uniformly to both internal and external ideas or values. Whereas a defensive skeptic typically applies this framework only to that which threatens existing beliefs or could introduce emotional risk. This leaves uncomfortable assumptions unexamined and is arguably just a form of intellectualisation rather than a legitimate desire to be more critically minded.

Are you fairly applying your skepticism to AI?

Defensive skepticism in practice

It’s likely you’ve experienced this before, perhaps it’s something you might upon reflection notice in yourself. Refusing to evaluate or approve AI tooling because “it represents too great a risk”, yet not choosing to understand it. Being critical of those that use AI as part of their workflow, not because their approach is flawed, but because they have an approach at all. Or perhaps you’ve seen the perpetual sandman. Not reading, not testing, not forming an opinion and treating that absence as a position. These all seem like unique and perhaps understandable responses but they all produce the same outcome: you’re not in the room when it matters.

The price of being defensive

Absence is not a defence nor is refusal. AI adoption is something that will happen, LLMs do not need to be correct to be successful. They just need to be close enough. Your developers are going to use AI, your HR staff are going to use AI and your executives are going to push AI. In security we frequently talk about Known Knowns, Known Unknowns and Unknown Unknowns. Usually it’s the unknowns that keep us awake at night. But here’s an unknown worth losing sleep over: In every room you’re absent from you’ve left an empty chair. That chair was your chance to help make decisions about how AI gets adopted, what guardrails should exist, what data should flow where. Those decisions are getting made right now, by those that show up. You’re on the back foot, responding not guiding, flailing not leading.

Philosophy meets practicality

Question. Why do you patch vulnerabilities? There are a lot of answers to this question, and the majority of them are correct. Consider for a moment, if you will, the futility of the practice. Do vulnerabilities ever stop? You don’t refuse to patch a vulnerability simply because there will be more. When we must, we simply do. Yet somehow this logic doesn't survive contact with AI. What you're doing every time you patch is what Camus described. You push the boulder knowing it rolls back. You don't stop because it's futile, you find meaning in the effort. Camus concluded this wasn't despair, it was engagement in spite of the absurdity. "One must imagine Sisyphus happy."

So why is AI the boulder you've decided to sit down in front of?

Being in the room

So where does all this leave us? The answer is clear, you need to be in the room. You need to be an active part of the decision making process. This does not mean you’re a cheerleader. You don’t have to be “pro-ai”. Rather you should let the very skepticism that is excluding you become a virtue. This means your solution to AI in your organisation cannot simply be no.

Are we post-CTF or is it simply time to rethink?

Mon, 23 Mar 2026 06:36:40 GMT

I've had the privilege of working on some amazing CTFs (Capture the Flags) in my career. From building internal ones, to delivering them for companies at cons or being part of the team running them at community events. Making challenges and seeing people learn from them is something that brings me great joy. There is a pretty insane amount of effort that goes into a good CTF. Coming up with challenges, spinning up reliable infrastructure for hundreds or thousands of people to use and wrangling a bunch of volunteers takes patience and perseverance and hundreds of hours usually crunched into a few weeks/days/hours prior to the event.

CrikeyCon 11 was no exception. We had big dreams about starting earlier but soon found ourselves working right up to the bell (even pushing challenges during the event). In a lot of ways I probably wouldn't change a thing about that, pressure makes diamonds.

As usual I want to thank the challenge writers (you guys rock) and my fellow organisers and volunteers (you know who you are). Your continued dedication and hard work is so impressive and something you should be proud of.

Without further ado, here is my retrospective for the CrikeyCon 11 CTF.

The ai-lephant in the room

For those that didn't attend CrikeyCon this year or attend the closing ceremony (where were you!?) you missed what could be considered a reasonably large disruption within the CTF space: AI. Now to most CTF attendees or organisers this isn't a surprise: AI is pretty well equipped to aid competitors. This year however, it definitely exceeded just aiding. We had our first competitor to simply authenticate Claude Code with their CTFd cookies and prompt it to complete the CTF. There was a lot more to it and I won't steal that competitor's thunder and will share/link them once they have written it up for themselves. For clarification we (the organisers of the CTF) were aware of this and largely saw it as a reasonable experiment. The surprising (or perhaps unsurprising to me) outcome was that this competitor won. The winning competitor gracefully passed on their first prize to the non-agentic winner. The agent basically flattened our CTF completing 79 out of 80ish challenges including access control challenges but excluding lock-picking challenges.

As with any disruptive technology there are different schools of thought as to what this means and I want to capture some of them for posterity.

CTFs are dead

This was probably the most common reaction from challenge writers. Writing unique challenges is hard and there is an expectation that people will put effort in to solving them. It's a natural reaction, one of the greatest rewards from writing a challenge is when you absolutely nail it. Nailing a challenge has a few different forms:

You make a challenge that gets solved by 1 team
You make a challenge that fits its category and difficulty rating perfectly
You make a challenge that is unique and quirky and absolutely frustrates the hell out of a team/player

All of these are what I personally consider success when writing a challenge. Sometimes your challenges are meant to be solved (especially 101 challenges), sometimes they aren't meant to be solved and sometimes you just really want the winning team to beg you for the answer.

The outcome on Saturday takes away all of these rewards. If your challenge can be solved by AI then every team has access to it, therefore point 1 doesn't apply. Point 2: the category and difficulty doesn't matter. And point 3? Well, Claude doesn't complain. This leaves less incentive for challenge authors to create challenges which then leaves the CTF scene less vibrant and interesting.

CTFs are going to have to change but I am not quite sure how

The other reaction was: hmmm, what do we do? I call this the what's next reaction. Essentially these people acknowledge the coolness of the technology and are also not sure how we can incorporate it. I think this is probably the second most popular viewpoint. They see that there is a future but aren't quite sure about how we get there. In some cases there is a feeling of not wanting to own the future. This is something I saw in some CTF players and challenge writers as well.

Wow this is cool / Wow this is scary

Pretty self-explanatory. Most people fall into this category. They are either excited or afraid. Either one is a perfectly natural response. I have a post I am cooking up about this topic so I won't spend too much time on it.

So what does the future of CTF look like?

These are just my thoughts so take them with a grain of salt.

We probably need two streams of CTF: a classic CTF and an agentic centric CTF

The traditional CTF still has a place and I think it should continue. The reason I say this is that the core benefit for participants (aside from Lego and bragging rights) is learning. We have a responsibility to learners as CTF writers to keep giving them a place to play and learn. That being said we will probably have to begin to take a harder line towards AI within these competitions and rethink our challenges a little bit.

During the CTF we offered prizes to people that would come up and admit/show us that they were using AI. We had at one point a line of 12 people waiting to show us laptops open with ChatGPT and Claude windows on display. AI is inescapable so we need to think about how we can account for that. This is going to take time and creativity.

The second stream is an agentic centric stream. I have some thoughts about what this might look like.

Agent jungle gym

It has long been a desire of myself and other CrikeyCon peeps to bring back a form of the Red vs Blue CTF challenge. These are dynamic attack and defence challenges that more closely imitate the way in which attackers and defenders work in a network. I think the outcome of this weekend just past justifies that. I've sketched out something I like to call the jungle gym. It's a hybrid between red vs blue and a king of the hill challenge with a bit of a twist: agents only, or a combination of agents and humans.

Essentially there are three rounds. At the beginning of each round you get to submit your agent/agents. That agent is presented with a series of boxes they are tasked with protecting and a series of boxes they are tasked with attacking. The boxes they are protecting have services and flags that must remain available; this is a prerequisite to score offensive points. Points are awarded for keeping your services available and taking down other teams' services. Each round is a novel environment and in between rounds competitors get to work on their agentic approaches before the next one starts.

This creates an interesting and dynamic environment for players and observers and also removes the inherent edge of AI. I am also exploring how to showcase what the agents are doing, and as to if it should be run by a form of AI dungeon master that handles injects. It would also be interesting to see if a team for example decided to not use AI and compare their placing to others. This is still early stages and we are a fair way out from it. My soft goal is to run an online version of this sometime in the later half of this year. If you're interested in taking part reach out to me, I will be writing and sharing more about this soon.

I will also be seeking sponsors to help pay for tokens as well so that there isn't a financial barrier to participation. If you've got someone that can help with that please point them in my direction.

AI has benefits for challenge creators

Writing challenges is time consuming. There are a lot of ways that we as challenge creators can use agents to improve this process.

I experimented a fair bit with agents this year and have the following observations:

Agents make much better UI/UX than we would normally, especially in web challenges

UI/UX historically hasn't been part of the challenge space. A lot of people that write challenges are not actively incentivised to provide a good UI for something that will be used for one day. Agents change this a bit. Having an LLM create a functioning user interface that goes over the top of your web challenge can be extremely powerful. This allows for much better visual storytelling and theming as well. We had some genuinely unique and interesting challenges that were greatly enhanced by the interface we could put on top of them.

Agents can add dynamism to challenges

Last year I experimented with agentic challenges by creating an IRC server that was populated with LLMs. Each LLM had different levels of protection from prompt injection and had a flag they needed to protect. This year we initially didn't have any LLM challenges, but during the day I set myself a challenge to use an agent to build an agentic challenge and push it live before the day ended.

The result was a challenge called slop-machine. Essentially, you pulled the lever and you got a random challenge displayed within an iFrame on the page. This challenge was live vibe-coded by an LLM (qwen3.5-35b-a3b, an open-source model) immediately after you pulled the lever and presented to you without a human reviewing it. Topics for challenges were mostlyweb and crypto, obviously limited by the implementation. There was an element of luck to the challenge in that you had a much greater chance of being presented with a challenge you literally could not solve (either because the LLM just borked it or the answer was wrong).

It was interesting to watch. Some people came up and asked why it was so easy, while I watched others pull and pull the lever getting perpetually wrong challenges. It's fun watching the non-deterministic nature of LLMs play out in real time. Dynamic interaction is a really powerful means of making an engaging and interesting challenge and I've got a lot more ideas that I will be bringing to future CTFs.

Agents are generally pretty capable of handling your 101 challenges

101 challenges are a must for any CTF, especially for Crikey. A lot of attendees are students or people new to the industry. These challenges are all very similar each year but still need to be slightly different and fit the yearly theme. These make up a reasonable portion of challenges each year and are still time consuming. Agents are great at making these (with a bit of prodding). So if you're building 101 challenges, agents are your friend. Avoid trying to one-shot multiple at the same time. I tried a mix of both and the one-shot multiple challenges were extremely unreliable. Use planning mode and be clear about what you want.

Agents are better at solving challenges than writing them

In a similar vein, agents are much better at solving challenges currently than writing them. Agentic output is inherently derivative; it's noisy enough to be interesting but not yet truly novel. Challenge writing requires novel thinking and current context that LLMs lack. Humans currently are still the best people to write difficult and interesting challenges. I would recommend avoiding using agents for anything beyond the easiest challenges, agent centric challenges, UI/UX, dynamic testing and writing IaC for infrastructure at the moment.

Alright, enough AI.

What we did right

Created a great vibe within the CTF room

CTFs should be social and communication/community reliance are super important skills. Participants are choosing to give up a lot of the benefits of a con by taking part so we want to try to bring them that experience. This year (thanks to our awesome community and sponsors) we had a bunch of prizes. This allowed us to come up with some interesting means of giving out prizes. Aside from the AI confessional we also had a rap battle. We offered 1000 points to the winner of the battle, giving them 45 minutes to come up with a unique rap. This was entirely off the cuff and honestly we did not expect the uptake; something like 8 teams came up and competed. We ended up having a tie for first and gave each team another 15 minutes to come up with some lines for a rap battle.

The finalist teams were both university cybersecurity societies. It was awesome to see the next generation of cybersecurity being so willing to get up in front of the community and have fun. I think that says a lot about the unique community we have in our industry and it's something I and the other organisers want to bring to future cons.

Improved infrastructure maturity

Last year we had a largely click-ops based approach to deploying our CTF infrastructure and I spent days making sure everything was working. This year was entirely different. Underlying infrastructure was handled by Terraform, scheduling services was handled by Nomad and challenge definition files were subject to linting and CI/CD. This wasn't perfect; we had a few little blips where we had to scale up the instances larger than we expected, but it made the process of getting started so much easier. Taking the learnings from this away, I will have a post soon where I officially open source the infrastructure template I've created. My hope is that this will help everyone run better CTFs themselves.

Some changes I want to make include creating traditional test harnesses as well as agentic ones, so that LLMs can have a crack at challenges. This will give us an idea as to whether our challenges are solvable via LLMs and also help spot issues with availability and performance earlier.

Fresh blood in the challenge creator space

I've said it a million times but I am so thankful for our challenge writers. I have an extra pile of thanks for our new challenge writers. You've become part of a strange dysfunctional family that is so much better for you being there. It takes a lot of courage to write a CTF challenge; you're basically asking people to hack something you made. We saw some awesome challenges from new creators that were fresh, vibrant and different.

What we could do better

Start earlier (we try)

My perpetual dream with every CTF is that I wish I'd started earlier. This is a pretty hard ask for our volunteers and creators so it's probably going to stay a wish. There are definitely project management and organisational style bits and bobs we could implement as well.

Expand our challenge base

Lots of great feedback this year about blue team and forensics challenges. These are things we'd love to have more of. Our challenge make up currently reflects our creators, that is very web/offensive sec heavy. Variety is the spice of life. So if you're interested in becoming a challenge writer please reach out to me. There is so much to gain from it and writing challenges is such a great way to validate understanding. We are open to any challenge from anyone and if you're looking for ideas we are also full of those.

Embrace AI

I've said enough about this above. There is more we can do in this space and we will.

Happy hacking and see you next year.

Is AI security really different from ProdSec?

Tue, 11 Nov 2025 04:37:34 GMT

Subscribe to get the next post delivered directly to your inbox

In the mid-to-late 1800s humanity developed the humble beginnings of evidence based western medicine, doctors treated illness using medication (mostly opium) and performed basic surgery. We began to see reductions in mortality rates and improvements in public health. Medical history of course, didn’t end in the 1800s, beginning in the 1870s new “medical” boom occurred: patent medicine vendors travelling the countryside pitching cures.

These cures weren’t for diseases that medicine was already addressing such as cholera or typhoid rather they were for entirely new ailments. Fun sounding diseases such as americanitis (the disease of living too fast), neurasthenia (exhaustion with technology) and hysteria (a dark stain on our medical history) became the modern plague. They weren’t pitched as evolutions of known diseases, rather they were new conditions that medicine couldn’t understand because it was “outdated”. Opium? How quaint, Surgery? How barbaric.

The new era of disease required a new proprietary fix with its own new secret ingredients. Vendors would publish almanacs, hold lectures and make charts flooded with medical-adjacent terms and classification systems. Suddenly everyone wanted, nay, needed Dr Kilmer’s Swamp Root or Lydia Pinkham’s Vegetable Compound. Doctors were dismayed, and pointed out that these new conditions were in fact just anxiety, poor diet or depression wrapped into a much better looking wrapper. This was dismissed by the public and the patent vendors as just them “not getting it”, they were out of date dinosaurs that hadn’t yet adapted to the reality of the new disease.

Why is this relevant? Well, patent medicines did sometimes contain novel active ingredients (usually boatloads of cocaine and alcohol). Some potentially might have worked. Instead of integrating these discoveries or understandings into medicine, vendors built entire parallel industries around them, replete with new diagnostic frameworks and certifications. This didn’t stop in the US until 1906 when the Pure Food and Drug Act forced vendors to disclose ingredients and stop making fraudulent claims.

Flash forward to the present, in the mid to late 2010s humanity developed the humble beginnings of functional AI. Whilst the concept for a large language model had existed for a while, the emergence of both the internet and affordable hardware enabled scale beyond our wildest dreams. The history of AI didn’t end in the 2010’s, in the early to late 2020’s we started to see these tools explode. The adoption of these tools without consideration or planning led to the development of an entire industry built around the need to secure these tools. This industry created its own taxonomy of threats with pretty charts and lectures designed to legitimise it. Prompt injection, emergent capabilities and model poisoning became all the rage.

This new era of threats required it's own proprietary practices, tools and teams. Suddenly everyone wanted what these vendors were selling. Security practitioners were dismayed and pointed out that these new conditions were in-fact just product and application security wrapped into a different, more appealing wrapper.

Why is this relevant? Well there is some truth to the idea that novel security challenges exist within AI, but it’s being packaged as if there is a need for it to operate separately from ProdSec and AppSec. There is an entire parallel industry being created when its really just an extension of our existing practices with some new learnings necessary.

The patent medicine industry for today’s ailment is inventively named AI security.

We’re in an AI boom/bubble/whatever, I am not going to debate about the validity or reality of it because it isn’t relevant. You, me and everyone else have got a job to do and that job most likely involves securing something that is getting or already has AI, regardless of our feelings or opinions. Technology being technology and hype-cycles being hype-cycles we have quickly found ourselves facing a petri-dish of half-baked immature technology smearing rapidly expanding lifeforms all over the place. We’re also continually being bombarded with the latest and greatest solution to fix the problems that tend to occur when half-baked immature technology gets unleashed upon business and individuals alike. What are these problems? Are they something unique? My argument (if it wasn’t already clear) is no and no.

AI Security is largely just ProdSec and/or AppSec. Note: I say largely, because there is legitimate necessity for researchers to focus on novel attacks that might arise within these models and a need to update our classification methodologies. Furthermore there is space for new approaches to solving problems and ways that we can use these tools to our benefits. But as a defender/blue/purple teamer I struggle to see the need for this to exist as a separate discipline.

AI systems are just software systems (usually APIs) with unusual/quirky characteristics. They take inputs, process them (through parameters as opposed to explicit logic) and produce outputs. The concerns these systems create map neatly to traditional product security domains. To clarify this I have outlined some of them below:

Prompt Injection aka Input Validation Failure -

Regardless as to if you’re sanitising SQL inputs or sanitising prompts to prevent injection, the pattern is identical. We have our methods for addressing this including:

Input sanitisation
Structured input formats
Delimiters between system instructions and user input
Filtering layers before processing (anyone heard of a WAF before)
Output validation
Principle of least privilege
Input length limiting and rate limiting

Jailbreaking aka AuthN/AuthZ Bypass -

Securing who can query your data and under what circumstances is the same problem whether its a model or an API.

System level guardrails independent of the model
Filtering layers before and after processing
Behavioural analysis for detecting policy violations
Principle of least privilege
Monitoring for known patterns (WAF again)

Model Poisoning aka Software Supply Chain attack -

A poisoned npm package and a backdoored model weight from Hugging Face are exactly the same threat category. Both are just third party dependencies

Model provenance tracking and signing
Model sourcing decisions
SBOMs
Dependency scanning
Code review
Integrity of training pipelines
Controlled training environments
Version control

This mapping continues all the way up and down the threat landscape and I might dedicate a large post to this alone.

So what's the prescription from this ProdSec lead? Extend your existing ProdSec/AppSec program to cover AI systems. Train your team on the quirks of probabilistic behaviour and adversarial ML, yes. Add some new tools for model scanning and prompt testing, absolutely. But don't create a parallel security organisation. Don't buy a vendor platform that treats AI security as fundamentally separate from the rest of your security posture. The patent medicine era ended when regulators forced transparency about ingredients. The AI security vendor boom will end when organisations realise they already have most of the ingredients in their existing security programs, they just need to apply them to a new (if quirky) type of software system.

The patent medicine vendors were selling sugar water with cocaine. The AI security vendors are selling you AppSec/ProdSec with buzzwords. Some of the ingredients are genuinely useful. Most of what you need, you already have.

Practical short term steps for managing supply chain risk in JavaScript/Python

Thu, 18 Sep 2025 02:12:45 GMT

Subscribe to get the next post delivered directly to your inbox

If you’ve got an app that even tangentially touches NodeJS/NPM (really if you have eyes and ears) you’ve probably seen these.

["Shai-Hulud" NPM attack runs malicious GitHub Action

An NPM attack compromised dozens of popular packages which then ran malicious GitHub actions in the compromised accounts

getsafety.com/blog-posts/shai-hulud-npm-attack

](https://getsafety.com/blog-posts/shai-hulud-npm-attack)

[ctrl/tinycolor and 40+ NPM Packages Compromised - StepSecurity

The popular @ctrl/tinycolor package with over 2 million weekly downloads has been compromised alongside 40+ other NPM packages in a sophisticated supply chain attack dubbed "Shai-Hulud". The malware self-propagates across maintainer packages, harvests AWS/GCP/Azure credentials using TruffleHog, and establishes persistence through GitHub Actions backdoors - representing a major escalation in NPM ecosystem threats.

www.stepsecurity.io/blog/ctrl-tinycolor-and-40-npm-packages-compromised

](https://www.stepsecurity.io/blog/ctrl-tinycolor-and-40-npm-packages-compromised)

[Nx npm Packages Compromised in Supply Chain Attack Weaponizi...

Malicious Nx npm versions stole secrets and wallet info using AI CLI tools; Socket’s AI scanner detected the supply chain attack and flagged the malwa...

socket.dev/blog/nx-packages-compromised

](https://socket.dev/blog/nx-packages-compromised)

I am not going to spend a lot of time writing about these particular pieces of malware. The posts above such as that from my mate Paul McCarty will do the job much better than I ever could.

I am also not going to talk much about how to respond to this directly. To those affected I wish you a relatively peaceful IR process.

Rather, I want to spend my words talking about how we as engineers, can move forward from this, providing some steps for short term solutions, plans for medium term solutions and then thought pieces about longer term plays.

Key thoughts:

Our model of dependency trust is broken and has been broken since the foundation of package managers
Industry standards for managing this risk are fundamentally faulty.
Manifest-based composition analysis is at best a great distraction from improving your security posture
Vulnerability management is less about managing vulnerabilities and more about managing expectations, CVSS/EPSS you name it, they are all largely awful. The number of CVE’s never gets smaller and your system never gets more secure.
Reachability analysis might give us some relief from this but I’ve yet to see if actually provide benefits to triaging vulnerabilities.

Anyways on to the actual practical bits, this post is going to examine short term solutions for JavaScript and Python in a very compressed manner. I will take a look at other languages in future posts.

Pinning dependencies

Dependencies are typically specified within two key file types, manifest files and lock files. Manifest files package.json, go.mod) typically contain the list of direct dependencies e.g. ones you have explicitly installed via npm install or pip install . Lock files on the other hand contain not only the direct dependencies but they typically also contain indirect or transitive dependencies as well as some form of cryptographic verification information (largely used to reduce downloading things that haven’t changed or preventing corruption).

The dependencies within manifest files are declared typically using a constraint format such as package_name@>1.2.3 this would indicate that we are seeking to install the dependency package_name with a version greater than 1.2.3 from a package repository.

Version tags as represented within most dependency repositories once published are typically available forever. There are circumstances where a package may need to be removed (such as cases of supply chain compromise) and the behaviour of the registries vary. You can see a breakdown of behaviours below.

NPM

Response Code: 404
Removal Policy: Limited unpublishing (24-72 hours after publish) plus other conditions
Alternative Actions: Tombstone/placeholder for established packages

PyPi

Response Code: 404
Removal Policy: Authors can delete releases but not entire projects easily
Alternative Actions: "Yanked" releases - hidden but metadata visible

Maven Central

Response Code: Never returns 404
Removal Policy: Never deletes - core immutability principle
Alternative Actions: Deprecation warnings only

RubyGems

Response Code: 404
Removal Policy: Complete removal possible but discouraged
Alternative Actions: "Yank" versions - hidden from search

Cargo

Response Code: Normal response with warning
Removal Policy: Cannot delete once published
Alternative Actions: "Yank" versions - unavailable for new projects

NuGet

Response Code: 404
Removal Policy: Complete deletion allowed for package owners
Alternative Actions: "Unlist" packages - hidden from search

We can see that some registries actually don’t have facilities for removal whatsoever rather they rely upon the behaviour of the package manager itself.

These behaviours combined with the typical constraint definition within manifest files create the first component of supply chain risk.

Going back to the package_name@>1.2.3 example. When all is working, a maintainer publishes 1.2.4 and you run npm install which ultimately results in you getting v1.2.4 of package_name. What happens if (like recent compromises) a bad actor publishes v.1.2.5 ? Well next time you run npm install you’re getting the bad version and its game over.

How do we fix this?

Provide strict constraints

Define your dependencies using explicit constraints rather than wide constraints. For example you should specify [email protected] rather than package_name@>1.2.3

This only protects your direct dependencies, unfortunately most dependencies have dependencies and they may not specify their dependencies in a manner that is as strict as yours.

Use deterministic operating modes

The largest cause of frustration and introducer of risk within dependency managers is when they operate in a non-deterministic manner, NPM is a great example of this behaviour:

Running npm install results in an interesting set of behaviours

Install performs a full dependency resolution which makes it slow. This process is necessary if you don’t already have a package-lock.json but is entirely superfluous if you do.
If dependencies (either direct or indirect) have loose constraints install will update them transparently without prompting. This is fine if you intend it, but can be a disaster when security is the aim.

Arguably this violates the entire point of a lock file but hey.

NPM offers an alternative install mode called ci which is designed to install exactly what is within the package-lock.json and will fail if it doesn’t match or can’t be installed.

So for NPM people:

Developers: Use npm install when adding or upgrading dependencies, then commit package-lock.json. Afterwards use npm ci
CI/CD & Production: Always use npm ci to guarantee a clean, reproducible install.

I’ve highlighted similar operating modes for other dependency tools within the Javascript/Python ecosystem.

Package Manager	npm ci Equivalent	Deterministic
yarn	`yarn install --frozen-lockfile` or `yarn install --immutable`	✅ Yes
pnpm	`pnpm install --frozen-lockfile`	✅ Yes
bun	`bun install --frozen-lockfile`	✅ Yes
pip	`pip-sync` (pip-tools)	⚠️ Limited (External tool)
Poetry	`poetry install`	✅ Yes
Pipenv	`pipenv sync`	✅ Yes
uv	`uv sync --frozen`	✅ Yes

Pythonistas: I recommend the use of Astral’s uv dependency manager. It’s just all around better than any other dependency manager from a speed and safety perspective.

Js/Ts Developers: I recommend the use of bun or pnpm. Again these are just all around better than the other options.

❝

Use deterministic operating modes of dependency managers combined with explicit version constraints to ensure you aren’t getting the latest versions of dependencies without explicitly seeking them.

Takeaway 1

Disabling or limiting lifecycle scripts

A common feature of dependency managers is their ability to run scripts through the lifecycle of the package install process e.g. pre/post install. These are also an extremely common attack path for malware authors as they tend to be overlooked by security tooling. Sources for numbers below will be provided in a research doc.

Javascript

Total Packages: 2.5M+
Lifecycle Hooks: 2.2% use install scripts (~55,000)
Malicious Usage: 0.9-1.3%

The Javascript ecosystem is the clearest case for disabling lifecycle hook scripts**,** several modern dependency managers such as bun and pnpm actually do this by default. They require you to explicitly enable each lifecycle hook for each dependency that has them. If switching dependency managers isn’t possible you are best served by disabling them via the npm ci --ignore-scripts command. If you’re in the rare situation where you need a script it is best to look at moving to bun or pnpm.

Python

Total Packages: 530K+
Lifecycle Hooks: ~33% use setup.py (~175,000)
Malicious Usage: 0.06-0.2%

The Python ecosystem is a little less clear, unfortunately there isn’t a direct guaranteed means of disabling pre-install scripts nor does the data really support it. There is the ability if you’re operating with a relatively well supported set of dependencies to force the installation of binary wheel formats rather than building anything from source.

Python as an interpreted language relies heavily upon system libraries get a lot of its work done. Some of this code needs to be built for specific systems but in most cases the interfaces they interact with are common enough across a set of systems (or the dependencies license permissive enough) that they can be pre-compiled and distributed as binary that will run just work. In the Python ecosystem these are called wheels a wheel is basically zip file that contains the python source as well as the binary extensions.

Using wheels means you don’t execute the setup.py script which closes down a vector of risk. You can force pip and uv to attempt to only use wheels by passing the --only-binary=all command. This will fail if a dependency you’re using doesn’t have a wheel though. In pip you can also pass a list of dependencies that you want installed only from wheel so this allows you to find the ones that have setup.py scripts and specify them. Luckily uv is just plain better and allows you to pass --no-binary=some-other-package package_name as well, so you can specify exactly the dependencies to install from source and just audit those for risk. If you combine this with the previous suggestion of dependency pinning you are narrowing the window of risk down to when you legitimately update the dependencies that you allow pre-install scripts to run in during a supply chain attack.

❝

Disable lifecycle scripts within the Javascript and Python ecosystems to limit your exposure to malware.

Takeaway 2

That’s enough for now, I will spend a bit of time on Java for the next post.

Pragmatic AppSec: Part 1 - A map that reflects the territory

Fri, 12 Sep 2025 04:47:31 GMT

Welcome to my new series: Pragmatic AppSec. It aims to provide security professionals at organisations that are AppSec curious with the necessary information to kickstart a program.

AppSec, whilst being extremely effective is frequently deprioritised, the success of it at an organisation is frequently dependent upon a single engineers willingness to drive it. It’s important to note that this series is based off my own experience and every business is different. The aim is to make this as agnostic as possible because AppSec is for everyone that has an App. I’ve been at large companies, medium companies and very small companies and have seen and implemented various versions of this. You will see me frequently talk about discovering business needs and appetites, this is an important aspect of any cybersecurity program: your solution needs to be right-sized to risk, resource and scale.

❝

Throughout the series you will see sections called Pragmatic Mantras these are designed to be thought-provoking callouts that help highlight useful tools of thought.

Pragmatic Mantra Example

Prerequisites

There is no specific profile of what makes a good Appsec engineer or leader. What you’ve done in the past just determines how much work you’ll need to do to give people the right information and support. That being said there are a few stereotypical archetypes within the Appsec sphere. I’ve outlined these below for posterity.

The former pentester: a lot of people in the field have worked in offensive security. This history shows in the way that they approach solving application security problems. I’d argue this strain of thinking dominates Appsec, sometimes to its detriment.

The compliance person: There is strain of compliance framework driven approaches to AppSec. This attracts a compliance centric archetype to the role. Again this background shows in the way the program runs and problems are solved.

There is nothing inherently right or wrong with these archetypes and their approaches will suit different businesses differently.

Fun fact: I don’t come from either of these backgrounds and a lot of the best Application security engineers I’ve met don’t either. To reflect this I figured I’d do my least favourite thing and talk about myself.

What is my background?

That’s a bit of a weird one, I started by studying tech at university and spent some time writing software. I very quickly discovered that I did not want to do that and ran to Systems Administration. This changed very early onto my career into DevOps and SRE and Security kind of just led from there.

The common thread woven throughout this is that I’ve spent a lot of time being close to engineers. DevOps and SRE was all about the product and engineering interface and Developer efficiency was my aim. My early security roles were all about securing developers either through building a product for them or working as a security engineer within Developer Platforms teams. Engineers productivity and safety was the ultimate deliverable for those roles.

As you will soon see successful Application Security requires working closely and considerately with engineers. Their productivity is the primary currency (aside from dollars) that we spend when we add security to a process.

❝

Engineers productivity is the currency of AppSec, spend it wisely

Pragmatic Mantra 1

Part 1 of this series looks at creating a common understanding of not only what AppSec is, but the establishment of an underlying map that helps guide you towards success. It’s not the most practical of the series but is certainly one of the most important. We are establishing the common language a.k.a the lingua franca of AppSec.

What is AppSec?

AppSec, rightly or wrongly, means a lot of different things to different people and that’s great! But for a program to work we need a common definition. Security historically was an afterthought, we built software and then worried about security. This is generally regarded as a bad idea, software is a brittle and fickle entity that will happily work its way around any badly structured controls. As the concept of SaaS became the predominate form of software consumption we quickly discovered through black eyes and breaches that we needed to change how we approached software security. Application Security or AppSec arose to fill this need.

If you asked most people what AppSec is the answer you will get is something along the lines of:

Application security is the practice of protecting software applications throughout their lifecycle by identifying, fixing and preventing security vulnerabilities through secure coding practices, automated testing tools, security reviews and runtime protection.

This is a great description of the kinds of tools and processes that are involved in an AppSec program but I would argue it really misses the crux of AppSec. Culture

I like to define AppSec as:

Application Security (AppSec) is the practice of empowering engineering teams to build secure software through architectural guidance and frictionless, well-integrated tooling that enhance delivery velocity rather than impede it. AppSec prioritises secure architecture and design guidance over tooling alone, with tools serving to support and execute that guidance effectively. Success depends on cultivating a security-conscious culture and communicating meaningful metrics tailored to both engineering teams (actionable technical insights) and management (business risk and program effectiveness).

Yeah, it’s a mouthful. I promise I will explain it better in the future.

A rationale for thinking this way

Now this definition might raise eyebrows, for starters, shift-left happened and we are still dealing with the side effects of that. Why are we focussing on developers? Shift-left and historical approaches like it made a series of mistakes when attempting to involve developers into the security cycle; for starters developers are not security people. The approach I advocate for in the above definition and this series corrects these mistakes by ensuring that responsibility still lies with the AppSec function.

We are engineers as well and therefore need to be at the centre or at the very least operating within the confines of the engineering function at an organisation. Our aim is not to make security developers responsibility, rather we seek to encourage them to build in a manner that is inherently secure. This aim is achieved through guidance, involvement, understanding and consideration.

Whenever your thinking about how to approach this:

Developers do not exist to fix our vulnerabilities they have a product to deliver. We need to enable them to deliver a product and do it in a way that addresses our vulnerabilities.
Developers are experts at writing software. We need to mirror this expertise and innately understand what they are doing.
Developers are overburdened and frequently under the pump. We need to be a pressure relief valve and encourage them to help us find pressure points.
Developers want to maintain their software and have a functioning safe development environment but frequently lack the ability or capacity to obtain justification for doing so. We need to help them get that justification.

The virtuous cycle of success in AppSec

At the core of any AppSec program you should have three core aims/principles/objectives/goals whatever floats your boat. I’ve created the following acronym as a memorisation method.

Mould, Advise, Publicise - M.A.P

Mould: Shape a security conscious, blame-free culture centred around collaboration that empowers engineers, whilst enabling you to ask for things when you need them.
Advise: Provide suitable guidance/advice/support and enable it with tooling that is velocity considerate and meets developers where they are.
Publicise: Expose and communicate success in a manner suitable to your audience - both engineers and leadership.

Each of these objectives sustain and nurture each other and will enable you to achieve whatever security/risk aims your company has.

What does this look like in practice? A virtuous cycle of success

The M.A.P that reflects the territory

Moulding a culture helps you identify opportunities to advise, these give you success to publicise which then encourages further cultural improvements. The self-sustaining nature of this cycle creates a positive feedback loop that ultimately drives you towards a successful AppSec program.

A practical example

AppSec Engineer Billie has identified that we have a series of vulnerabilities within container images at the organisation that are persisting across multiple sprints. Billie has invested a lot of time with Engineering Team A.

Through conversations and pair coding sessions she has identified that the reason they struggle to patch vulnerabilities in their containers is because they are just too complex. Billie has provided the engineers with a safe space to talk about frustrations and can readily map her needs to theirs.

Billie offers to help them build better containers, focussing on the developer experience in communications. “Let’s make these easier for your team to iterate on these so that you can test our new versions of X language”

Through iteration and work (yes she might be actually helping make a container) Billie has helped produced a container that not only aids the developers in their aims but also reduces the attack surface and vulnerability numbers substantially.

This is communicated to engineering leadership as a win for developer productivity, to security leadership as a win for security and to engineers as a win for making their job easier.

Engineering Team B enjoying the benefits of Engineering Team A’s new containers, reaches out to Billie and asks her to help them with their build pipeline. Billie has identified that vulnerabilities within CI/CD are not being readily maintained.

And so on…

Thus the virtuous cycle sustains

Alright that’s enough for now

In future posts we will examine each of these pillars in detail and look at how we can build them out but for now keep this model in mind as we delve deeper into this.

Thanks for reading

Don't forget to do your damn job and tell people about it

Thu, 14 Aug 2025 14:00:00 GMT

A good friend of mine who runs a great Appsec company, sent me this great article about a month ago. It wasn't new to me, but in recent weeks it's really been playing on my mind.

This hit its peak when I gave a guest lecture at QUT to a class full of people looking to get into tech and cybersecurity. It was a great opportunity and I am extremely grateful for the chance to help guide the next generation of engineers.

The job climate is obviously a little whack at the moment so the usual set of questions came up:

How do I get into X job?
What can I do to stand out in an interview?

The kinds of advice I fall back to here beyond what they might be already doing are typically things like:

Have a portfolio
Learn in public
Broaden your skills
Focus on the market and the needs of the businesses in it

It struck me after this lecture had concluded that a lot of these behaviours are exactly the kinds of extracurriculars mentioned in this article.

These are the things that result in people getting stuck going from mid to senior and above roles (in an individual contributor space). The irony of this thought occurring while I was doing exactly the kind of activity that's the subject of this article is not lost on me. Nor is the fact that I am writing about it in a blog article.

It's a curious industry we work in (cybersecurity), the path to success in our roles is usually circuitous and tangled. The ability to demonstrate your impact when you're slowly cutting your way through the jungle is vital to your long term success, it's also a skill that a lot of people lack.

The most common method (at least from what I have seen) to make up for this shortfall is to focus on doing everything that isn't your job. You're part of the community, you're writing blog posts, maybe you maintain some open source projects. There is nothing wrong with that, we need people to shape communities, maintain software and share their thoughts; but you need to do your damn job be able to communicate the impact it's having.

The idea of both actually having impact and communicating it is something so thoroughly missed in both formal education and informal education. It's also arguably why a lot of big tech companies have such awful approaches to role changes (10k word role change docs read by people that you've never worked is so good right?). Forcing people to write large documents that justify their existence gives people that aren't doing their job, or lack the skills to demonstrate impact, the ability to spend time pretending they are doing their job. The need to create the appearance of impact also creates a whole graveyard of barely functioning PoC's propping up large tech companies. A lot of these same problems still exist in smaller contexts, the difference is that people just don't get promoted/maybe lose your job and/or the size of teams gets larger than it needs.

Here's the rub, I've always found it incredible just how much time people in security engineering roles have to write thought-pieces. Again irony, I am an engineering leader writing a thought piece blog. For full transparency, finding time write a blog is like herding cats for me, my participation in various groups and programs is a carefully negotiated dance and the rare occasion I get to do anything new it requires several rituals to all known gods.

Enterprise security is rough, product security is frequently non-existent and we are always talking about just how threats never stop coming. We speak constantly about how we always need more people. All of this is to say that if you're taking the time to write a blog or run a community group you should consider whether you're doing your job and demonstrating your impact. This is a super vital time to consider this, the world is changing and regardless of your position on AI it will have an impact on how people do their jobs, the types of roles that are available and the quality of work people expect.

So ask yourself an honest question am I:

doing my job
demonstrating the impact

If the answer is yes to both, then great, thank you for your time.

If the answer is no to either of these, well here is what can you do?

Do your damn job
Measure your damn impact (if you don't know how I am going herd some cats and write posts for you)
Talk about your damn outcomes
If you've still got time left over do the rest

Getting comfortable in the grey part 2 - Becoming a generalist and transitioning careers

Thu, 29 Feb 2024 07:04:32 GMT

This post is the second in a series designed to examine the value of generalists in cyber security. This post looks at how one might transition to be a generalist in security. Future posts will be targeted at constructing teams and other topics. Check back each Thursday for the next part.

A disclaimer: there is no single path into cyber security, no algorithm or bootcamp that can get you there. It unfortunately takes a combination of luck and passion. This post won't necessarily get you a job but it might help you get closer to one.

How does one get comfy in the grey? How does one become a cyber generalist? It's a weird question to answer for me and honestly it took me a while to think about what I could possibly tell someone that wants to move into cyber security as a generalist. The truth is that it never crossed my own mind, I was incredibly lucky and my own passions happened to align really well with the needs of the industry. However, I strongly believe that cyber is a better place when we have a more diverse and my own journey does highlight some really valuable advice that could be key for someone looking to make a change.

So despite my own declaration that there is no algorithm for getting into cyber I will attempt to boil my experience down to something similar to an algorithm. A set of rules for getting comfy in the grey.

1. Start elsewhere Entry level jobs are extremely rare in security. Unless you're graduating and are seeking a SOC role or are lucky enough to make it into a graduates program or traineeship it can be really hard for someone to go directly in to cyber. So rule number 1 and perhaps the most important part of being a generalist is that you should start elsewhere. Cybersecurity ultimately protects people and businesses and a key aspect of being able to do that is to understand what you are protecting. Therefore a good place to start is to look at what kinds of threats we care about right now and seek a job in those areas. This could be a role in DevOps, Software Engineering, Machine Learning or Infrastructure Engineering. These roles will do a few things: a) they let you explore your passions, b) they give you the chance to identify and resolve security issues that you can then develop into resume defining items and c) they get you working. Arguably, most of the iconic people in security today didn't start in security (it didn't exist) they appreciate diversity and understand that security comes from all places.

2. Don't do "cyber" certifications first Unless you have a very specific role in mind certifications aren't really it. Does this mean that you shouldn't do them? No, but arguably the ROI on most certifications isn't going to be there early in your career especially if you can manage to get yourself a role where other (relevant) certifications such as the AWS vendor certs could be covered by employers. You should focus on getting relevant experience in the job role that you're starting in. This could be taking part in your companies security champions program, helping the company identify vulnerabilities (especially relevant for Software and DevOps engineers) and making sure you're recording and writing down each of these experiences so that when the experience issue comes up you're able to identify key aspects of your career that answer those questions.

3. Do make your own tools Tooling isn't everything in security, but it certainly helps out a lot. Identify problems you can solve and write code to fix it. Coding is a skill you need in security, I disagree strongly that you aren't advantaged by being able to write and design software in security. Even GRC peeps should be writing tools that enhance the delivery of their work. This doesn't mean they need to be writing a scanner, but they certainly should be creating python scripts or other tools to make data visualisations that will wow executives. Write tools that solve your problems, make those tools open source and get people using them. Why? This will give you exposure to solving problems and vulnerabilities that arise in the day to day maintenance of a product. Ultimately that is what you do in a lot of security roles. You will be constantly identifying and potentially solving problems. If you understand how this works already you are a huge leap ahead of the competition. Furthermore you will have a name! Imagine going to a job role and someone interviewing you uses a tool that you wrote daily. This happens more than you realise.

4. Seek startups and scale ups It isn't easy getting a security role at an established organisation if you don't have experience. So start small, get a role at a startup or a scaleup especially one that is very early stage and in an industry vertical you are interested in. This role could be a hybrid role, it could also just be a software engineer or infrastructure engineer. A lot of startups give you a lot of latitude and freedom in the early days. This means you can solve security problems as part of your day job without risk. Furthermore, most startups do not hire security people (although they probably should and a future post will cover this) so there is a distinct chance for you to move into a security role just as the organisation grows. Startups are also the place I have learnt the most, the risk of sink or swim is daunting but it gives you an incredible clarity and freedom that you cannot get elsewhere. Embrace the grey that is inherently part of being an early startup employee and you will be well on your way to getting comfy in the grey.

5. Convert experience into relevant experience Finally and perhaps the most important rule: Experience using the above rules is only valid if it's relevant. This means you need to be able to translate what you're doing into security relevant experience. Highlight the bugs you've found, identify the improvements you've made and speak the language of risk. Security is part of every job role and you can easily highlight the things you're doing in a way that is relevant to the role you're seeking. If you can't? then you need to find yourself a way to do that. This means that you need to seek opportunity (again this is why startups are great) to grow in your current role. Inversely to this, admit when you don't know something or don't have experience. Lying isn't going to get you anywhere, integrity and ethical behaviour is a key part of being a security professional and admitting that you don't know something isn't immediately a failure. No one; I repeat, no one understands everything nor is it realistic for people to expect all applicants to know everything. Cultural fit and passion are important markers for the kinds of employers that you want to work for.

Alright, that is enough for now. Check back next week for more. Leave a comment in the social channels if you have any feedback.

Getting comfortable in the grey part 1 - Generalists and job roles

Tue, 20 Feb 2024 02:01:32 GMT

Patty: I'll be the good guy.
Shermy: I'll be the bad guy.
Patty: What are you going to be, Charlie Brown?
Charlie Brown: I'll be sort of in-between; I'll be a hypocrite! - Charles M. Schulz

This post is the first in a series designed to examine the value of generalists in cyber security. This first one looks at how we can get our hands on more generalists and why we should. Future posts will be targeted at people looking to transition as well as constructing teams and other topics. Check back each Thursday for the next part

Specialisation is important... or is it? It's a question I have frequently asked myself as a self admitted generalist. What should my speciality be? Do I need a specialisation? Is it too late for me to pick one? The further I got into my career the more frantic this question got and the less obvious the answers got, the greater the pressure I felt to make a decision, it became a burning issue.

Until it didn't. As I moved from Sysadmin to DevOps/SRE, to Security, to DevSecOps I increasingly came to realise that the biggest differentiator for me was my lack of formal "specialisation". My broad based approach allowed me to fit into and lead any team. I could speak the language and walk the walk of developers, security engineers and everything in between. Most importantly though, I never found myself in a position where I felt the need to block with the standard security ban hammer. I also built solutions just as comfortably as I guided and protected.

I got comfortable in the grey. I also believe that more people should. I've noticed a trend whereby we focus so much training content and pathways upon two core concepts: GRC and Pentesting. This leaves us with a dismal gap, we've got a whole lot of people that know how to translate vulnerabilities into business risk and a whole bunch of people that are great at finding vulnerabilities but a significant shortage of people that can practically fix them or better yet avoid them. This is where the precis of my argument lies: we need a whole lot more people that sit right in the middle, we need more generalists.

This isn't an attack on specialists, we need them just as much my ideal team is a mix of both. It's more an argument that generalists create the possibility to address a few core issues within our industry

Hiring differently to fix the gap problem

Finding vulnerabilities isn't easier, but its a hell of a lot easier than eliminating a whole class of vulnerabilities. This requires a unique combination of knowledge. Eliminating classes could be as simple as changing a framework or library but it could also involve deep infrastructure changes or mental model shifts. Generalists are the best people to guide and construct this. They understand enough of a broad set of domains to be able to target the specialists towards the most effective and considered fix.

Generalists can readily transition from other parts of the tech industry

Layoffs are happening as we face a decades overdue realignment between financial expectations and reality and whilst its terrible and my heart goes out to those that are facing this; there is opportunity in adversity. Some incredibly talented software engineers, infrastructure engineers and more are looking for roles right now and it's highly possible they would make exceptional security engineers. We should embrace and bring them in, get rid of these ridiculous 5 year experience role requirements, get rid of the must have experience in blah. Unless you're looking for a principal that needs to be able to mentor people the most important thing we need is people that are keen in securing the world and interested in sharing their knowledge.

Diversity of thought greatly enhances our ability to identify new threats

Imagine having access to a never ending stream of people that are exposed to the pointy end of our software and infrastructure every single day. Now imagine that you could build a team out of those people! You get to have the people that you're securing or write the software you're securing working to secure it with you. The value of this is impossible to overstate. They might not have the language that we speak but they sure as hell know what they are talking about.

If you have any thoughts feel free to reach out to me on the various social channels.