Context: This post is the content of a lecture I delivered to Specialist Trainee Doctors in Anaesthesia from the North West of England at the NWRAG (North West Research and Audit Group) QI & Research Day on the 24th October 2016.
I want you to ask yourself – could I do an experimental clinical study?
In this post I use my current RCT, the TERSC study, as a case study for discussing various concepts related to developing ideas into trials. For full details about the TERSC study have a look at our protocol published in the TRIALS journal.
Simon Sinek’s TED talk “How great leaders inspire action.” which followed from his book “Start with Why” is worth a watch – it’s had over 28 million views.
In it he explains the simple concept of the “Golden Circle”. It goes like this: Most organizations work from the outside in, that is they tell you what they do, some know how they do it and very few know why they do it. Great organisations work from the inside out – they know their why first. He uses Apple as an example saying if they were like other computer companies they would say: “We make great computers, they’re beautifully designed and easy to use. Want to buy one?”. Instead they say: “Everything we do we believe in challenging the status quo. We believe in thinking differently. The way we challenge the status quo is by making our products beautifully designed, simple to use and user-friendly. We just happen to make great computers. Want to buy one?”
When I reflected on this concept I realized the Golden Circle could also apply to how we develop and deliver research studies.
Let’s start with “Why”. What fascinates you? What makes you curious? Have you ever had an academic itch to scratch? Something you needed to know, a burning curiosity? That could be your “why”. Of course your “why” may simply be getting something published and enhancing your CV. This is perfectly acceptable but without a stronger “why” you may struggle to push through the tough bits.
I suggest exploring what your clinical purposes and passions are for your career, and develop your research around those – they will form your “why”. Mine over the years have been a focus on introducing system-wide changes in the clinical areas which I believed would have the largest impact on patient outcomes e.g. sepsis care bundles in 2005, hospital track and trigger system redesign, low tidal volume ventilation on ICU, health care associated infection care bundles in 2005 and enhanced recovery programmes for all major surgery since 2008 to name a few. More recently, aside from my own research, it has been to increase my organisation’s overall research output as the Director of R&D.
In 2013 I heard a colleague from Exeter talk about his experience with an intervention called Rectus Sheath Catheters (RSC). RSC’s provide analgesia for major abdominal surgery with an open midline approach and their data suggested it was potentially a reasonable alternative to thoracic epidural analgesia (TEA). I contacted him for details about insertion and management of these and proceeded to introduce them into my practice. Very soon it was clear to me that this was an effective technique e.g. when I went to visit one of my patients, called David, on the ward the morning after his major abdominal operation I found that he’d already eaten his breakfast and had been walking around the ward! In fact he required another similar abdominal operation a month later and requested me to insert the RSC again and we had the same great results. Despite this there seemed to be very poor uptake of RSC.
Indeed there is a marked clinical variation in analgesic techniques for major abdominal surgery within departments, within regions, and nationally, with seemingly earlier adoption of various abdominal wall block procedures in the South of England compared to the North. This sort of clinical variation within a region was measured in the South West by the SWARM trainee research network in a survey of practice. As there were no RCT comparing RSC to TEA this became my “why” for the TERSC study.
More recently my inspiration for possible perioperative innovations and research projects have stemmed from my various hobby interests:
- Sports medicine e.g. a current RCT to compare Superstarch to standard PreLoad pre-operatively to reduce insulin resistance and consequently morbidity,
- Biohacking e.g. use of HRV to predict deterioration postoperatively and Wim Hof breathing techniques to reduce immune activation and consequent inflammation peri-operatively
- Nutritional science e.g. reduction in fatty liver pre-liver resection surgery using LCHF diets and reduction in inflammation with the same.
Part of your “why” will include the awareness of the wider benefits to you, other staff, the patient and your organisation of being involved in research.
Here are some of the benefits generic to all clinical research activity and some specifically to the TERSC study example:
- Improved patient outcomes regardless of group allocation
- Improved hospital outcomes for hospitals that are more research active
- Attract higher calibre staff in the future
- Research experience
- Chief Investigator (CI) experience
- Project management experience
- Create a research culture
- Create structures for perioperative research delivery
- RSC training
- RSC packs
- Departmental profile
- Lecture invitations
- Research Grants and funding
- Academic collaborators
- Peer review invitations
- Trial Steering Committee invitation
- Appreciate value of mixed methods research
- Value of Patient and Public Involvement (PPI) and their potential as focus groups for other projects
- NIHR Portfolio accrual benefitting other roles.
So work out your “Why” – this will give you inspiration for your research ideas and maintain your enthusiasm.
Focus, discipline & resilience are prerequisites to push through the inevitable bumps, but make sure you also think about strategy and tactics to ensure you succeed without blowing up along the way like Johnny here. Of course the actual moral of that video is that if you give a South African half an opportunity he’ll take it!
To do this you need to identify your strength & weakness e.g. having a vision and project management are my strengths but I know I’m weaker at statistics and I don’t enjoy the reporting aspect – some of these I’ll work on e.g. practice my academic writing skills, but others e.g. acquiring high level statistical skills are not worth the time consumption when I have access to statisticians.
Develop your leadership skills and be able to delegate effectively. This requires serious investment in your teams. Collaborate and surround yourself with experts – some of them will become mentors. Volunteer to be the local Principal Investigator (PI) for some large multicentre studies. Not only will this be an extremely educational experience regarding how such large studies are designed, protocols written and delivered, but will connect you with the CI’s of these studies who may become one of those collaborators (as has been the case with TERSC). Attend relevant national research meetings where you will be able to network gaining further collaborators and mentors. In my experience our academics are all very approachable and helpful. The key meetings are the UK Critical Care Research Forum & the Peri-operative Medicine Clinical Trials Network conferences which include sections where you can present your study idea and get feedback from the experts. I would encourage registering with the latter organisation as a PI or Local Investigator (LI) to receive some training and put you in position to be offered a PI role for one of the large studies.
Get some research education. There are several good Research Methodology courses available. There is a comprehensive generic 5 day course provided by the combined universities of the North West which takes place twice a year (April & September). Single days can also be attended. The 2 day Edinburgh course is specifically aimed at anaesthesia and critical care research and held every November.
Learn how to tackle the approvals process and understand the HRA system. This will save you a lot of frustration in the future. You will need to register on IRAS and work through the on-line tutorials to get permission to undertake any research. The key is a well written detailed protocol which you can paste into many of the domains.
Try to develop research that aligns with your consultant job plan as much as possible to make it more manageable. My job plan as an example has evolved over a 10 year period whereby the clinical components consist of a CPET clinic, critical care and core anaesthesia lists of major colorectal and hepatobiliary surgery. This aligns perfectly with my research topics and my leadership of the Enhanced Recovery Programmes. I am able to see some of my research patients in the clinic and help enroll them into trials, subsequently anaesthetise them and also follow them up on my critical care days. I do a session as Specialty Lead for Anaesthesia for the NIHR GM CRN and another session as R&D Director for my Trust. Both of these research managerial roles are synergistic with my personal research in a variety of ways.
Balance is something I’ve got wrong in the past so I’ve worked hard at applying the Pareto principle i.e. 20% of your invested input is responsible for 80% of the results obtained. The key is identifying the 80% of your time which is gaining you very little output and ruthlessly eliminating it. That means strategically saying no to some requests.
Spend time developing systems and structures up front that will maximize your future efficiency e.g. I use these standardised templates for all my study protocols, outcome measures and reporting. Use all the help you can get for study design and advice on funding options and possible grant applications. The NIHR North West Research Design Service (RDS) provide this for free including connecting you with appropriate academics to be co-applicants.
I use a variety of productivity tools to streamline my workload e.g. Inbox zero; E-mail batching with delayed sending (for Mac mail client users); Rescue Time; Evernote (this is a game changer with wide application); Dropbox (allows easy collaboration, co-authoring and document sharing) and work off-line while using the Pomodora technique for Deep Work.
If you implement these tips together with any you discover you’ll maximise efficiency but as the great Peter Drucker said:
“Efficiency is doing things right; effectiveness is doing the right things.”
In other words only apply your effort and efficiency to tasks you’ve identified via the Pareto Principle as generating you the most output. This will make you maximally effective.
Ultimately getting your “How” right will allow you to spin multiple professional plates while not neglecting time for family, fun and staying as fit and healthy as possible – this is the pathway to long-term success.
The “What” for me is the methodology of the specific study and the lessons you learn during the delivery. We learn more from our mistakes than our successes so I’ve learnt plenty from each of my studies. Probably the main one for TERSC is not to do another complex intervention analgesic study!
Start with defining the objective of the study. The aim of this study was to assess the efficacy, safety and acceptability to patients of RSC.
The acceptability was measured using a nested qualitative study which is quite novel in anaesthesia research. It will give a balance between the traditional quantitative outcomes and the patients’ views – which is ultimately what we should base our choices on.
We included two sub-studies which measure the plasma concentration of the local anaesthetic agents and markers of stress response in the same 20 patients. We used convenience sampling (in order to avoid blood sampling out of hours) to recruit these patients but ensured a balance from each study group and between types of surgery.
The framework of this study is a randomised parallel group concealed allocation non-blinded superiority trial with a nested qualitative study of a subset of patients. As the study involves two active comparators we would only need to demonstrate non-inferiority due to some of the other advantages of the new intervention however this would require a much larger which would not be feasible to recruit to from a single centre – see “lessons learnt” section below.
Our study population are adult patients undergoing open midline major abdominal surgery who are willing to consent. We exclude those with allergies or contra-indications to the interventions, other incisions, chronic abdominal pain, opiate tolerance and ano-rectal surgery.
The interventions were a standard TEA and RSC. Importantly we are comparing an optimal TEA to an optimal RSC. Here is a link to a video I produced of a RSC intervention on a patient with a BMI of 40. Note that this study is not merely about the comparison between these two primary analgesic techniques but rather a comparison of two entire complex analgesic packages and that both groups experience an optimal enhanced recovery pathway.
The primary outcome was the difference in mean pain score (VAS) on movement from supine to sitting position at 24 hours after extubation between groups. I chose this as it is a key marker of the ability to mobilise early and has been used in other similar studies.
Secondary outcomes include:
- Further comparisons of analgesic efficacy and effectiveness e.g. VAS at other time points & the evolution thereof, time to 1st opiate & total opiate, sleep quality VAS, categorisation of functional quality of analgesia i.e. mobility and breathing & an overall satisfaction score of analgesia;
- Functional recovery comparing PQRS scores from baseline, time to mobilisation, return of GI function; achievement of ERP mobilisation goals, time to meeting discharge criteria and hospital length of stay;
- Safety & morbidity are measured by fluid balance, incidence of hypotension, nausea & vomiting VAS, incidence of ileus and a day 5 POMS with any surgical complications within 30 days classified for severity according to the Clavien-Dindo scale.
- Only short-term cost-effectiveness will be calculated limited to assessing procedural costs (equipment and time) and length of stay in critical care and ward beds.
Sample size was calculated as 60 patients per arm based on achieving 85% power to detect a 10 mm difference in the primary end point, from 40 mm in the TEA group to 30mm (SD=18mm) in the RSC group, at the 5% level (two-sided t-test). A 10mm change has been reported previously as the minimum VAS change that was clinically significant (MCID). The mean and standard deviation for the TEA group was taken from large metanalyses. There is no published RSC RCT data on which to base a mean VAS thus we have postulated a MCID improvement in mean VAS. As these are both active interventions there is unlikely to be any difference in the efficacy (for working interventions) but we are assuming there will be lower failure rates for RSC (6% has been reported in a large observational series whereas TEA failure rates of 30-40% are reported in the literature) which may influence the primary outcomes via the intention to treat analysis.
We are utilising multimodal approaches to recruit patients. Research nurses attend Cancer MDT for awareness of any eligible cases and attendance dates. This also embeds them in clinical teams raising culture and awareness. Pre-operative clinics are attended. Patient and clinical specific TERSC posters are in all clinics. The Waiting List Clerks are regularly contacted to search for cases and the theatreman system is interrogated twice a week for any missing cases. All surgeons and anaesthetists who do pre-operative clinics are aware of eligibility criteria. I produce a recruitment certificate for clinicians whose patients have been recruited to use for their appraisal folders. And the list goes on…
I spent substantial time working with the team at Imperial College CTU to design the TERSC study specifics for their online InForm database system. This performs the randomisation function providing allocation concealment. Our allocation sequence generation uses 3 surgical strata & 2 age strata as these groups would have expected differences in the secondary outcomes and need to be balanced between the groups.
Data collection and management also make use of this online system with source data from information collected in theatre, patient diaries, PQRS questionnaires and case notes. It also provides on-going governance and is auditable regarding any data changes or manipulation ensuring this study conforms to the highest levels of research quality.
- Difficulty in recruitment prediction at trial design due to evolving surgical approaches (open cases are increasingly transverse incisions rather than midline & retiring open surgeons have been replaced by laparoscopic surgeons).
- Impossible to model the impact of opiate tolerance, chronic abdominal pain & midline scarring on rrecruitment.
- Superiority versus Non-inferiority. Using superiority is a gamble as very difficult to show a difference between 2 active comparators. Non-inferiority would be ideal as a RSC only needs to be no worse than TEA on the primary outcome however the sample size is not achievable in a single centre and this complex intervention was not feasible to conduct in multiple centres at the time it was designed. Hence the superiority approach was a pragmatic choice and as the reported rate of epidural failure from larger studies is much higher than the rates reported from observational series for RSC it was considered reasonable. All the secondary outcomes together with the patient acceptability data from the qualitative study will help to interpret the results.
- A functional primary outcome e.g. length of hospital stay would have been more relevant to me however it would have required a much larger sample size to show even a 1 day reduction due to the already short stays for colonic surgery.
- The resource intensity of recruitment across clinics – I was naive to the geographically spread of the clinics these patients attend. This realisation has informed other current and future perioperative studies.
- The workload involved in delivering the RSC intervention with limited practitioners – two colleagues who were proficient during the design phase were not available during delivery (one emigrated and the other on maternity leave). The vast majority of RSC were inserted by me & one other colleague. I had hoped to train other colleagues during the study but the number of RSC available to train each individual to the level necessary for study patients has been insufficient. This is due to the large number of different anaesthetists covering all of these lists and half the potential RSC training cases being randomised to TEA. Training will be easier after the study concludes.
- Training of recovery and ward staff in managing the RSC and delivering the LA boluses on time via the AmBit pumps has been challenging, particularly as the colorectal ward was changed during the study. This led to many new nurses to train. The use of bank staff on night staff also resulted in some RSC patients not receiving boluses. This issue has led to some patients with RSC losing faith in the intervention when they perceive staff lack of confidence in using the pumps which is likely to affect their analgesic experience due to the psychological component (the qualitative study has highlighted this).
- In view of the difficulty of training staff to deliver the RSC boluses on time via the pump that continuous infusions may have been a better option. I had decided on boluses over continuous due to my clinical experience, & that of others, with both options prior to the study however it has not turned out to be pragmatic and has also introduced the other variable that the RSC pain score will be influenced by the proximity to a bolus whilst the epidural is not.
- In practice the incredible subjectivity of VAS as an end point is concerning. It is well validated and used for similar studies but seems so dependent on each patients’ frame of reference. In future I would consider instead one of the composite scores e.g. OBAS if a functional primary outcome wasn’t feasible from a recruitment point of view.
- Rather than selecting a single VAS pain score time point for the primary outcome I would rather perform time point averaging over all 6 time points. This would smooth out the influence of the timing of RSC boluses and any missing data and be more representative of the entire time course of both interventions. We will be reporting this as a secondary measurement.
- The resource intensity of data collection – the fixed time points for VAS scoring allows little flexibility and frequent late operation finishes has led to very difficult time points for data collection. For future studies I would sacrifice scientific accuracy for pragmatism and collect pain data twice a day without being specific on times. This would allow our Acute Pain Teams to contribute to some of this data collection for the research teams during their routine rounds.
In the ideal world I would have conducted a feasibility or a pilot study to tease out most of these issues. However the NIHR grant I was awarded had put out a themed call for surgical studies that year giving me my best chance of success with the application. These grants are incredibly competitive with far lower probability I would have received the award by delaying the application in order to perform a feasibility study. Ultimately a necessary tactical decision. In addition a feasibility study of this nature would still have required a research nurse to deliver it. These are not available without NIHR portfolio status and funding. A chicken and egg scenario.
First identify your “Why”, then develop your “How” (your generic structure improved over time) & finally your “What” (how you actually deliver a specific project).
Keep the design as simple & pragmatic as possible to increase the likelihood of completion.
Using the TERSC study for illustration probably hasn’t enthused you to do your own study, so instead I offer two further examples of studies I’ve completed which would have been feasible for any of you.
The first was an RCT which compared 3 different sedation techniques for ERCP. I completed this while I was a specialist registrar back in 2002. I recruited, consented, delivered the intervention and collected all the data during my allocated research day. I understand the current training programme in anaesthesia still allows a research day if a study of sufficient quality is undertaken.
Another was a prospective haemodynamic study I managed to deliver entirely during my clinical sessions on ICU as a consultant without any help apart from a few of the research nurses pitching in towards the end.
These demonstrate that clinical research is feasible within a conventional clinical training programme and consultant job. Models for non-academic research career pathways are being keenly looked at by the NIHR Anaesthesia National Specialty Group and the NIAA. The Association of UK University Hospitals have also produced a useful document regarding SPA/PA allocation which can be used for annual job planning allocation.
So, you CAN all do an experimental clinical study!
I would love some feedback in the comments section…