Agile in science: managing research projects with lessons from product development

1- Introduction

In the dynamic and ever-evolving landscape of scientific research, the adoption of agile methodologies signifies a transformative paradigm shift that promises to reshape the way we approach and conduct our science projects. Agile methods give scientists a dynamic framework to prioritize adaptability, collaboration, and efficiency.

Traditional research methods in sciences or conventional research methodologies have carved a noteworthy path toward scientific discovery. These methods include hypothesis construction, systematic observation, data collection, interpretation, and validation. These cyclical approaches are typically time-consuming labor-intensive, and might not always offer dynamic adaptability for spontaneous changes during the research. They also operate in a sequential pattern, and if any discrepancy happens at one stage, it requires one to return to the initial steps, consuming valuable resources and time.

In this post, we'll discuss the synergies between agile methodologies in science and their triumphs in product development. We'll talk about the fundamental principles of agile, its advantages over traditional research management, and its possible implementation for making research team progress more efficient and streamlined.

2- Evolution of agile management

Agile management, initially a child of the software development industry, emerged in the late 20th century as a response to the dynamic, volatile nature of technology development. Agile's core premise lies in iterative development, continuous adaptation, and flexibility in the face of change—concepts that contrast with the linear, inflexible frameworks of traditional project management. It offered enhanced productivity, reduced time-to-market, and improved product quality, enabling organizations to respond to market changes quickly.

With software development witnessing unprecedented success with Agile, other domains began to notice. Over time, Agile's principles made their way into various sectors, including scientific research. The expansion came from the realization that Agile's iterative process, feedback mechanisms, and short-term goal-oriented sprints could benefit the inherently uncertain, exploratory nature of scientific projects. It offered a revised perspective on scientific project management, allowing flexibility to modify experimental design mid-course, enabling rapid responses to new data or unexpected results, thus improving research efficiency.

3- Emergence of agile in research management

Agile for the scientific and research field operates similarly to Agile's application in software development. It blends this methodology with research practice, delivering a more flexible approach to scientific investigation. It refocuses the research process from rigid, pre-planned steps to a fluid, consistent learning journey where changes are welcome and adaptation is possible and promptly implemented. It is not unusual for scientists to adapt to changes based on experiment outcomes, which involve on-the-fly adjustments based on results. However, applying the agile methods introduces a heightened level of organization and structure to the process. Implementing agile strategies, such as Scrum, Kanban, and Lean, has driven research progression with enhanced speed and efficiency where requirements and solutions co-evolve through adaptive planning, early delivery, frequent reassessment, and efficient communication. These methodologies bring effective backlog management, encourage iterative progress, and promote constant contact and feedback, thus ensuring results can be delivered faster and research can adapt swiftly to unforeseen changes without derailing the entire project. Applying this agile approach to science can help scientists better navigate the dynamic nature of lab experiments by streamlining scientific inquiries and speeding up the trajectory of discovery, all empowered by teamwork and, ultimately, improving the quality of research outputs.

4- Agile methods in science: Breaking down research

Let's break down research into bite-sized pieces—like sprints in product development. It's about achieving the goals and decomposing large-scale, complex research problems into manageable, small-scale research cycles. It is a bottom-up approach, where small, iterative experimental cycles aggregate towards substantial research findings. Scientists can zoom in on specific tasks, get feedback, and keep the research process going. It is a series of interactive loops of 'Plan, Do, Analyze, Act' cycles. It is dynamic, and with each iteration, the research plan is updated based on what is learned from the previous process. At each stage, results are analyzed, variations are accounted for, and the research direction is optimized based on real-time data, feedback, and progressive learning. This continuous, evolutionary cycle of planning, execution, reflection, and adaptation provides a pathway to tackle the complex, dynamic world of biological science research. With this in mind, the agile research methods will allow:

Iterative Development: Each mini cycle is complete and can produce beneficial insights, making the research process more efficient. Each iteration serves as a checkpoint and an opportunity for reassessment, adjustments, and redirection, if necessary, based on the latest findings and analysis. Being responsive to changes keeps the research relevant, eliminates wasted resources on non-viable paths, and paves the way for potential breakthroughs that rigid plans may miss.
Adaptability: It's the capability to respond to change – changes in hypothesis, changes in experimental conditions, and even changes in research direction. This adaptability is particularly useful in research, where unexpected findings are common, allowing research to veer in new directions. The key is to thrive on feedback, permitting timely alterations to the research process.
Teamwork: Each member plays an adaptive yet autonomous role. The fact that all team members are part of the same agile sprint allows more minds to be engaged in problem-solving, creating a wider pool of ideas and potential solutions. Shared project visibility enables all team members, regardless of rank or specialization, to stay apprised of the overall project direction and progress. This way, decisions are made collectively and guided by each team member's expertise. This cross-functional collaboration also minimizes information silos, making for a truly integrated research team.
Improved project management: This is possible by keeping research projects on track, within budget, and in line with the set timelines. Using tools like the Kanban board or Scrum framework, teams can visualize the whole process, track the progress of individual tasks, prioritize and re-prioritize tasks, identify bottlenecks, and adapt to any emerging changes in real time. This real-time responsiveness to changes optimizes not only the process but also the results, reducing the chances of failure or unnecessary work.

5- Adopting Agile methods in research: Bridging science and product development

There are many tools to implement the agile methodology: Kanban boards scrum frameworks are one of them. They help visualize workflows, track experiments and let teams collaborate seamlessly—even if they're miles apart. Adopting agile tools in research involves tailoring the sprint cycle to suit the dynamic nature of scientific inquiry. Unlike the standard 2-week sprints in software development, research sprints may need an extended duration of 3 weeks to 1 month to accommodate the intricacies of scientific tasks. At the start of each sprint, researchers define their core question or objective, outlining functions like data collection or experiments needed to achieve the overarching goal. Setting clear deliverables for each assignment prevents perpetual research and allows for adaptability in the face of unforeseen changes. The sprints course will be like the following:

Sprint Planning: Sprint Planning is a crucial collaborative "ceremony" in the research project, involving the entire team to ensure alignment on the sprint's duration and content. Tasks are assigned to various team members, considering the diverse expertise needed. This inclusivity extends beyond researchers to include individuals handling financial procurement, ethics, and other support functions vital to the research process. During Sprint Planning, the team collectively identifies sprint objectives, outlining specific tasks like data collection, experiments, or literature reviews. Tangible deliverables and clear metrics for each task are established, providing a roadmap for the team to work towards during the sprint. This collaborative approach ensures everyone is on the same page, fostering efficiency and a shared commitment to the research goals.

Sprint Execution: During the Execution phase, team members actively engage in their designated tasks, diligently working through the entirety of the sprint duration. Each member contributes expertise and effort toward accomplishing the specific objectives outlined during the Sprint Planning. The collaborative effort involves a dynamic exchange of ideas, feedback, and support as the team tries to meet the overarching sprint goal. Throughout this phase, constant communication is critical, with team members regularly updating each other on progress, challenges, and any adjustments made to ensure alignment with the sprint's objectives. The focus is not just on individual task completion but on the collective advancement toward the defined sprint goal.

Scrum Updates: Regular scrum updates occur, typically every week. In our experience, conducting scrum updates twice a week for a duration of 15-20 minutes has proven effective. The format involves a roundtable discussion facilitated by a special "sablier" (hourglass) that allocates 3 minutes for each team member to review the board. The update structure consists of each team member answering key questions: (1) What did you accomplish since the last update? (2) What do you plan to accomplish before the next scrum? (3) What, if anything, is impeding your progress?

Team members might use this time to ask for help from colleagues, ensuring that everyone is aware of potential roadblocks and can contribute to finding solutions. Additionally, it provides a forum to share crucial information, such as the status of tasks that might impact others' work. This could involve simple but essential inquiries, like checking with the lab manager about the arrival date of antibodies and confirming their delivery status. Such details are vital for the seamless continuation of planned experiments, and this collaborative communication ensures that everyone is on the same page.

Review and Retrospective: The review and retrospective phase after each sprint involves a comprehensive examination of completed tasks, acknowledging successes, and openly discussing areas for improvement. Effectively, each completed task is reviewed, and the team assesses whether the deliverables were met. Team members reflect on what went well, identify challenges, and collaboratively formulate lessons learned. It is also an excellent opportunity to celebrate success and results. The documentation of such information serves as a valuable resource for onboarding new team members and maintaining a collective understanding of the project's iterative progress.

Next Sprint Planning: The next sprint begins with a new set of objectives, tasks, and goals, building on the insights gained from the previous sprint.

What is Scrum? | The Agile Journey with PM-Partners — https://www.pm-partners.com.au/the-agile-journey-a-scrum-overview/

This iterative process allows flexibility, adaptability, and a systematic approach to progressing through the research project, ensuring that the team can respond effectively to the evolving nature of scientific inquiry. Sprints can encompass diverse activities, such as literature reviews, with precise goals to avoid getting stuck in continuous reading without progressing one's research. Scrums, though still applicable, may find a balance with weekly updates, fostering team cohesion and understanding. For larger projects, breaking teams into smaller units, ideally consisting of three to six individuals, streamlines the scrum process and enhances overall efficiency.

6-Agile approach: Empowering teams, not micromanaging

One of the fundamental principles of agile methodology is that it is not synonymous with micro-management. Agile embraces a philosophy that empowers teams and individuals, fostering a collaborative and self-organizing approach to work. Unlike traditional management styles that may involve detailed oversight and strict control, Agile trusts teams to make decisions and adapt to changes in real time. In agile, the emphasis is on communication, collaboration, and delivering value. Teams are given the autonomy to decide how to achieve their goals best, encouraging creativity and innovation. Instead of being closely monitored, team members are trusted to self-manage and collaborate effectively. This approach recognizes that individuals closest to work often have the best insights into accomplishing tasks efficiently.

7-Conclusion

In a nutshell, bringing agile into the scientific playground isn't just about following the trend. It's a strategic move towards a future where breakthroughs aren't stuck in rigid structures. By breaking down research into doable tasks and using nifty Agile tools, scientists can kick up productivity, amp up collaboration, and roll with the punches. One common false belief is that agile methodologies are only applicable to software development and cannot be used in scientific research. Agile methodologies can be adapted to work with research however it is important to note that while agile methodologies can be adapted for research projects, there are some modifications that need to be made to the agile philosophy to better integrate it with traditional research methods. It is important though to ensure that team members have the skills, knowledge, and mindset to work effectively in a hybrid environment. Finally, agile isn't about micro-management, just operational clarity. Like a leader in product development who knows the end game, scientists diving into agile get the scoop on how each piece fits into the research puzzle. It's about understanding and being aware of the nitty-gritty of the process without letting go of their creativity.