I helped design and launch six Atlassian Professional Certifications between 2015 and 2017 while working at Adaptavist, an Atlassian partner. These weren't trivial multiple-choice quizzes. They were rigorous assessments designed to test whether someone could actually administer Jira, manage Confluence, or architect workflows in production environments – the kind of exams where you needed a couple of years of real hands-on experience to pass. Here's how it happened and what I learned about building certification programs that matter.
Why Atlassian Needed a Real Certification Program
Before this project, Atlassian had no standardized way to verify that someone claiming "Jira Admin" skills on their resume could actually do the work. Partners hired consultants who said they knew the tools. Enterprise clients needed proof that their admins could handle complex configurations. And individual professionals wanted credentials that meant something in the job market.
The gap was real. Atlassian's ecosystem was growing fast, with thousands of companies adopting Jira, Confluence, and the rest of the suite. But growth without quality assurance creates problems. A bad Jira admin can tank a team's productivity. A misconfigured Confluence instance becomes a graveyard of orphaned pages nobody can find.
Atlassian needed certifications that separated people who genuinely understood the tools from people who'd watched a few YouTube videos.
How Each Certification Was Designed: Working with a Psychometrician
The foundation of every certification was a blueprint designed in collaboration with a psychometrician from Alpine Testing. For each certification, a selection of people from Atlassian partners would meet with her in person for several days to work through the blueprint – defining what the exam should test, how questions should be weighted, and what separated a passing candidate from a failing one.
Atlassian wanted exams that tested real-world knowledge – the kind of thing that made it hard to just study and memorize for. You actually had to have been working with the tools for a couple of years to pass, especially the administration exams. The insight that we as partners brought was that real-world experience. Having spent years working with real clients on real Atlassian implementations informed the kinds of scenarios and decisions the exams tested.
After the initial blueprint was designed, I took over. The psychometrician from Alpine Testing would run the beta test of each exam and give me feedback on how individual questions and answers performed, as well as the overall pass/fail rate. I'd revise questions based on that data and develop further study materials.
Six Certifications from ACP-100 to ACP-500
The program launched six certifications, each targeting a distinct role in the Atlassian ecosystem:
ACP-100: Jira Administration
The flagship certification. This covered project configuration, permission schemes, workflow design, custom fields, and the day-to-day work of keeping Jira running for an organization. The exam tested scenario-based decision-making, not just feature recall. A question might describe a permission conflict and ask you to identify the root cause – the kind of problem real admins debug weekly.
ACP-200: Confluence Administration
Confluence administration is a different animal from Jira. It's about space management, permissions hierarchies, macros, templates, and helping teams actually find what they've written. The certification reflected those distinct challenges rather than copying the Jira exam format with different product names.
Advanced Workflows Badge
Workflow design in Jira is where simple administration becomes engineering. Conditions, validators, post functions, scripted transitions – this badge targeted people building complex automated workflows that drive business processes. It was the most technically demanding assessment in the program.
ACP-300: Agile Development with Jira Software
This one sat at the intersection of tool knowledge and methodology understanding. You had to know both how Scrum and Kanban boards work in Jira Software and why specific configurations support specific agile practices. Pure tool knowledge wasn't enough. Pure agile theory wasn't enough. You needed both.
ACP-400: Jira Service Desk
Service desk administration has its own domain – SLAs, customer portals, queues, automation rules, and the integration between ITSM practices and Jira's capabilities. This certification tested the unique skills required to run a service desk operation, not just general Jira administration applied to a different project type.
ACP-500: Sysadmin
The most infrastructure-focused certification. This covered installation, upgrades, performance tuning, database management, and the operational side of running Atlassian products at scale. The target audience was IT teams responsible for keeping Atlassian instances healthy across Data Center and Server deployments.
Writing Exam Questions: Why I Ended Up Doing Most of the Heavy Lifting
Writing good assessment questions is harder than most people realize. A poorly written question tests reading comprehension, not subject matter knowledge. A vaguely written question lets both experts and novices arrive at the "correct" answer for completely different reasons. And in Jira specifically, there's the added challenge of accidentally writing questions where there are multiple right answers – Jira lets you do almost anything multiple ways, so you have to be careful about the answers and distractors.
The certification development process involved subject matter experts from multiple Atlassian partners. But here's something that always differentiated Adaptavist: we only hired people with five or more years of Atlassian experience who really knew their stuff. We always brought our best to the table. Other partners, while respected and doing good work, often sent the people for whom they couldn't bill much – the less experienced people who had availability. Their contributions often had to be redone. I ended up rewriting a significant number of their questions, answers, and distractors.
Atlassian eventually worked solely with me for developing the training materials because the other partners just weren't able to contribute the same level of value. I don't want to diminish the contributions of others – there were definitely other people involved. But I ended up doing a lot of the heavy lifting.
The beta testing process used item analysis statistics – difficulty index, discrimination index, distractor analysis – to evaluate every question. A question where experts and novices both get it right isn't testing anything useful. A question where experts get it wrong might be poorly written or might be testing something that doesn't actually matter on the job. Questions were cut aggressively based on the data.
Building Study Materials That Filled a Massive Gap
Certifications without study materials create a frustrating experience. But the study materials for these certifications couldn't just point to existing training, because the existing training didn't cover enough.
The first thing I did for every certification was map out every topic the exam covered. I put this into a spreadsheet, went through every question, and wrote down what we were testing on. Then I cross-referenced that against Atlassian's existing Atlassian University training materials. Using Jira Administration as an example, I found that only about 25% of the topics in the certification exam were taught through the Jira Administration class.
This gap existed because Atlassian's instructional designers at the time didn't actually know the Atlassian tools that deeply. They were hiring instructional designers from Oracle who never fully learned the Atlassian products, so the training they produced stayed shallow and focused on the basics. That disconnect really showed up when it came to building the certification program.
I had to make up that gap through the certification study materials. Because we wanted these certifications to test real-world experience, I decided to write the study materials as case studies, inspired by the book The Phoenix Project. Candidates would read through a realistic scenario covering the exam topics, then work through a series of questions based on that case study. When I taught at conferences, I broke these out into slides and talked through them in day-long sessions so people could prepare for the exam.
The beta testing programs served double duty. They validated the exams statistically through the psychometrician's analysis, and they generated early advocates. Beta testers who passed became some of the most vocal supporters of the certification program because they'd seen firsthand that it tested real skills.
What I Learned About Technical Certification Design
Three lessons from this project stuck with me and have shaped every training and assessment project I've done since.
Real-world experience is what makes certifications meaningful. Atlassian products have hundreds of features. Most administrators use a fraction of them regularly. An exam that tries to cover everything becomes a trivia contest. An exam built on what practitioners actually do day-to-day – informed by partners who've been doing the work for years – becomes a meaningful credential.
Feedback loops in assessment design are non-negotiable. Without beta testing and item analysis, you're guessing about question quality. I've seen certification programs skip this step to save time and money. They end up with exams that frustrate candidates and don't predict job performance. The statistical validation phase isn't optional – it's where bad questions get caught before they undermine the program's credibility.
Certification programs are products, not projects. They need ongoing maintenance as the underlying products evolve. A certification that tested Jira Server knowledge in 2016 needed updates as Jira Cloud matured. Building the program was the beginning, not the end.
The Impact on Atlassian's Partner and Customer Ecosystem
These six certifications became the standard credentials for Atlassian professionals worldwide. Partners used them to validate consultant skills before client engagements. Enterprise customers used them in hiring decisions and internal career development paths. Individual professionals used them to differentiate themselves in a growing market.
The program also created a feedback loop for Atlassian's product teams. When large percentages of experienced administrators consistently struggled with specific product areas on exams, that signaled UX problems worth investigating.
FAQ
How were the Atlassian ACP certifications developed?
Each certification started with a blueprint designed in collaboration with a psychometrician from Alpine Testing. Subject matter experts from Atlassian partners met in person for several days to define what each exam should test, informed by years of real-world experience working with clients on Atlassian implementations. I wrote and rewrote exam questions, answers, and distractors, developed case-study-based study materials to fill gaps in existing Atlassian training, and worked with the psychometrician's beta test data to refine the exams. Every question went through statistical validation before the exams launched.
What made the Atlassian certification exams different from other IT certifications?
Having experienced partners inform the development made a real difference. The exams presented realistic scenarios that required candidates to apply knowledge the way they would on the job, informed by the real-world experience of people who'd been administering Atlassian tools for clients for years. This meant you actually needed a couple years of hands-on experience to pass, especially the administration exams. Pure book study wasn't enough.
How many Atlassian certifications were in the original program?
The program launched with six certifications: ACP-100 (Jira Administration), ACP-200 (Confluence Administration), ACP-300 (Agile Development with Jira Software), ACP-400 (Jira Service Desk), ACP-500 (Sysadmin), and an Advanced Workflows badge.
Related Case Studies
- Creating Atlassian University's JIRA Training Videos – The video training courses that complemented the certification program.
- The Atlassian Ecosystem Podcast – How I made Atlassian product knowledge accessible through a different medium.
- Learn for Jira: Building In-App Training from Concept to Sunset – Another approach to closing the Atlassian skills gap, this time with in-app training.