Thematic analysis for critical success factors

on next and what we will work on later. I think it's very, very compatible with a must-do, could-do and should-could-do’. I13 further elaborated on the Company’s planning strategies with the following statement: ‘So me putting out this timeline doesn't make sense because there are so many unknowns. So, we only take the time factor into account when all but the majority of the unknowns are addressed, and that's pretty much what you see in the planning, and that's three months’. In other words, release planning is almost unused by the ‘Case 4 - Retail’ company.

The company uses its roadmaps as inputs for more detailed planning, and PI planning and backlog prioritization are the mechanisms the company uses to align short- and long-term planning perspectives (roadmap). ‘Case 4 – Retail’, therefore, supports proposition P4.

use of different kinds of backlog categories (‘So for instance, there are only UI features. I can refine them with mobile team, web team. There are backend features only with backend developers’), having several categories of backlog items (such as ‘technical debt’ in addition to features.

6.2.2. Continuous plan monitoring

This theme is about review sessions in which different kinds of Product Owners and roadmap managers, etc., sync-up with each other to look at the roadmap and the backlog items in order to check the status of the project, correct the roadmap, and align items between teams.

These meetings follow a similar pattern and happen either every week or at the end of a Sprint.

Interestingly, these are not typical Scrum or Agile meetings, and they are rarely mentioned in the literature. Interviewees of Cases 2, 3, and 4 described having weekly Product Owner sync-up meetings to review dependencies, check on the roadmap items and the roadmap progress, and share problem-solving strategies. Another meeting type that was consistently present in all four cases was the Sprint review meeting. During these meetings, roadmap progress is presented. ‘Case 4 - Retail’ interviewees discussed incorporating roadmap progress into their current project’s formal Sprint result reports. The interviewees reported that communications regarding long-term planning happen via a number of channels, and that Product Owners, Scrum Masters, and stakeholders directly participate in such meetings. Monitoring via different channels gives those working on a project the opportunity to steer the project in a specific direction and either prevent or bridge the planning gap:

‘Senior stakeholders, if they see us deviating from strategy or the things that they have on their radar, that's their opportunity to tell us like, hey, guys, hang on a minute, maybe you should not look at this, but maybe you should look at that. So, by communicating

our direction, we give other people the opportunity to tell us to maybe adjust our direction’ (I13).

6.2.3. Tools usage

Tools usage was highlighted primarily by the interviewees of the ‘Case 1 – Bank’

company. Case 1’s Product Owners appeared to be struggling with tooling that is inadequate for managing the company’s complex roadmap. One ‘Case 1 – Bank’ interviewee stated the following: ‘I think we don't have a common view of all the POs in the squad’. Another ‘Case 1 – Bank’ interviewee mentioned that ‘The biggest drawback that we have is the required management tool that we are using. Instead of JIRA, we use ServiceNow, and it's a very limited tool with which you cannot really adjust a lot of things there or do proper boarding or do proper planning’. The opinions expressed by the ‘Case 1 – Bank’ interviewees regarding tools usage were not shared by the interviewees of any of the other cases. The ‘Case 3 – Food Retail’

reported using a combination of Jira and Excel; the Case 2 uses several tools, including JIRA;

and the ‘Case 4 – Retail’ uses the AHA tool to develop its company-wide roadmaps and the JIRA and Miro tools to plan at the team level.

6.2.4. Horizontal alignment and dependencies management

Cross-team dependencies and the timely delivery of commitment dependencies is a theme that emerged from each case. Sometimes the dependencies and stakeholders are not even known during the planning process, which can create a gap further on in the development process. I11 stated:

‘there is sometimes misalignment or contradiction if you have a dependency with other teams … For us the feature is really, really important. But then for another team,

because they work on a different portfolio, a priority is different, the [feature] is not important for them. So that's something that happens quite frequently, to be honest’.

One Product Owner felt that the root cause of this issues was a lack of an overarching priority list that spans across different portfolios. The Product Manager for the Case 4 company was critical of ASD’s ability to solve problems relating to dependencies: ‘I would say dependency of teams on each other is something that I think the Agile framework doesn't really, really address’. The Product Manager also stated that ‘I always, always try to make the ask as small as possible because, you know, a team is more willing to commit to one day of work than ten days of work … don't ask for the Ferrari if you can also have the FIAT’. The ‘Case 1 – Bank’ company’s interviewees reported that they hold a quarterly event designed to align dependencies across all teams. The ‘Case 3 - Food Retail’ company interviewees reported that they manage dependencies with PI planning. The Scrum Master of the ‘Case 2 – Media’

company puts Communities of Practices–cross-teams with expertise in a particular topic (e.g.

APIs or Front End technologies)–in the spotlight to solve the dependency challenge: ‘Because of the communities of practice, you know what the other team is also working on ... You share, you talk about it … I believe, are really beneficial to those long-term goals’. According to the shared product owner of the ‘Case 1 – Bank’ company, the attention of management is very important when working to solve dependencies: ‘I think this is a management role.

Management really plays a very strong part in getting all those stakeholders committed’.

6.2.5. Team engagement into road-mapping

Roadmap creation is typically done by the product owner or manager. However, without the input of the teams, the roadmaps that are generated may contain unrealistic plans.

The roadmaps may also lack information about significant efforts, dependencies, and even the feasibility of roadmap items. I05 discussed the issue of roadmap accuracy in the following

terms: ‘We see the gap … in our estimations. When we just defined the roadmap, we give very high-level estimations, and then we start working on it. We discover more impediments, risks.’

The interviewee also noted that ‘We started involving tech teams, and it's also briefing them

… asking for their feedback. So that's … giving them a better understanding of the tech teams’.

The ‘Case 2 – Media’ interviewees clearly noted that there had been times when their Agile teams had not understood where the roadmaps were coming from: ‘Previously it was a problem because nobody understood where these … roadmaps came from or they didn't see that’.

Managers in the ‘Case 2 – Media’ company is improving on this issue by involving technical leads into prioritization sessions and explaining goals and roadmaps to Agile teams. I10 shared a similar observation: ‘The big gap is due to too small stories; sometimes teams don't know the reason of this story and the result of this story”.

6.2.6. Advanced prioritization

One tactic that can prove dangerous or counterproductive is putting pressure on a short-term plan when the stakeholders have packed the long-short-term perspective with varied and unrealistic asks. ‘Case 1 - Bank’ Product Owner shared that, for the long-term projects and major initiatives, the ‘the number of stakeholders or the amount of work is not refined … it's very huge. So the initial estimation is really far [off]’. ‘Case 3 – Food Retail’ product owner shared details about pressures that can be felt by people on the requesting side as well: ‘There's always pressure … they always want it tomorrow, which will never happen’. ‘Case 2 – Media’

Product Owner called this a ‘roadmap rush’: ‘we need to produce more, and more, and more this roadmap rush’. The interviewees shared ways to manage pressure, including applying advanced prioritization techniques and managing the backlog. The thematic analysis suggests that the short- and long-term planning gap can be at least partially mitigated by the existence of well-considered long-term planning and well-considered roadmap prioritization processes.

In ‘Case 2 – Media’ and ‘Case 3 – Food Retail’, these processes were quite advanced multistep processes that combined the judgments of Product Owners with data and/or business considerations. For example, the I05 shared that ‘Using WSJF really helped because otherwise, everyone wants their own feature’. In addition to using the WSJF technique, I05 noted that the company validates every roadmap ask with a business case: ‘So people actually proof. What's the value in it? Why should we do it? And the validation with business cases to get the help from data analysts helps to kind of validate the idea’. The ‘Case 4 – Retail’ interviewees noted that their company uses business value sizing. The ‘Case 3 – Retail’ company also uses business value sizing, which is demonstrated in section 6.1.2.