Consultation with Parliament in connection with the Review of National Outcomes

Consultation Analysis Summary Report 

Executive Summary

The statutory Review of the National Outcomes seeks to answer the question “Do the National Outcomes still reflect the kind of Scotland we want to see?” and to collect evidence on gaps within the implementation of the National Outcomes across Scotland, referred to as the “implementation gap”.

We answered the review question by collecting and analysing evidence from four evidence strands: desk-based research (covering 34 sources), a public written consultation (87 responses), a stakeholder call for evidence (125 responses) and expert stakeholder meetings (110 stakeholders attending 11 meetings). 

Thematic analysis of text-based responses was completed manually. Each piece of evidence was analysed and coded through the lens of our review question “Do the National Outcomes still reflect the kind of Scotland we want to see?” 

The final themes and sub-themes were split into three categories for distribution and further use: 

1. National Outcomes and Indicators (4,809 comments covering 78 themes)
2. Implementation Gap (874 comments covering 11 themes)
3. Other (345 comments covering 12 themes)

Analysis showed that views were diverse, with over 5,000 comments categorised into more than 100 themes and 800 sub-themes. Although the NPF generally provides a useful high-level vision with good comprehensive coverage of key issues, there are gaps and areas which could be strengthened to better reflect the kind of Scotland we want to see. Key themes which are underrepresented within the current National Outcomes include: recognising the importance of care, good quality transport and housing, shifting focus towards a wellbeing economy, reducing inequalities, improving mental health, and prioritising climate action to combat climate change. 

Analysis of evidence on the implementation gap of the National Outcomes noted several common barriers including policy coherence, a complex reporting landscape, difficulties embedding the NPF in practice as a driver of changes, and dissatisfaction with current funding models. There was support for the development of a Wellbeing and Sustainable Development Bill as well as calls for increased accountability mechanisms including clarifying roles and responsibilities, better scrutiny and improved government transparency of actions taken and progress towards outcomes.

Once analysed, themes, and frequency of occurrences, were distributed across the NPF unit for consideration in the development of the new NPF National Outcomes, associated National Indicator Set, implementation plan and Wellbeing and Sustainable Development (WSD) Bill.

Introduction

The statutory Review of the National Outcomes seeks to answer the question:

Do the National Outcomes still reflect the kind of Scotland we want to see?

The review also seeks to collect evidence on gaps within the implementation of the National Outcomes across Scotland, herein referred to as the “implementation gap”.

We answered the review question by collecting and analysing the following evidence: 

• Desk-based research

Reviewing existing citizen engagement exercises, community action plans, collating additional evidence, and undertaking the discovery phase for the Wellbeing and Sustainable Development (WSD) Bill focussing on the implementation gap;

• Written consultation

Published through Citizen Space, designed to be accessible for members of the public;

• Call for evidence

Where organisations and experts can submit their view;

• Expert stakeholder meetings

To focus on specific areas (e.g. homelessness, care) as well as the implementation gap (one of the Finance and Public Administration Committee recommendations following their inquiry into the NPF) and engagement with the Scottish Youth Parliament.

This document outlines the methodology and results from the data collection and analysis of the four evidence strands by the National Performance Framework (NPF) Evidence and Reporting (E&R) team. Resulting themes will feed into the development of the new NPF National Outcomes, associated indicator set and NPF implementation plan.

Methodology

Data Collection

Data was collected from the following four evidence strands: desk-based research, a public written consultation, a stakeholder call for evidence and expert stakeholder meetings. 

The purpose of the desk-based research was to understand what stakeholders and communities have said during the myriad of other engagement and consultation opportunities and how this can be used to support any recommendations for changes to the NPF. The 34 sources included within the desk-based research have been described in Annex A.

The purpose of the written consultation was to establish the views of the public on the suitability of the National Outcomes as they are. The written consultation was hosted on Citizen Space following the SG guidelines for consultation best practice from 14th March to 12th June 2023. This consultation was aimed at the general public and included a consultation information pack to support users to complete the consultation with all the information they needed on the purpose of the NPF, the current National Outcomes and the aim of the consultation. The six questions included within the written consultation have been described in Annex B.

The purpose of the call for evidence was to allow stakeholders to submit their own research or evidence demonstrating what they would like to see in the new NPF and what needs to happen in order for the NPF to impact on their work. The call for evidence was launched at the same time as the written consultation and ran from 14th March to 12th June 2023. This evidence could have taken many forms, from short paragraphs to multi-page reports, to letters of support for topics, to requests for adaptations to the NPF via emails. The five questions provided as guidance for call for evidence respondents have been described in Annex C.

A series of 11 stakeholder meetings were held between March and June 2023. These sessions were themed and followed a similar workshop format as set out below. In addition, one open session was held with no theme. Most were facilitated by NPFU officials. 

The themes covered by the stakeholder meetings were: 

• Young people (facilitated by the Scottish Youth Parliament) 
• Care and Care Experience
• Business 
• Transport 
• Housing/homelessness 
• Environment (facilitated by the Economy and Environment Leadership Group) 
• Sustainable Development Goals 
• Gender 
• Consumers (facilitated by Consumer Scotland) 
• Just transition 

The format of the workshops covered three main areas: 

• What is good and what should be changed about the National Outcomes 
• Why does the theme matter to wellbeing 
• How should it be reflected in the National Outcomes 

Data Analysis

Each piece of evidence was given a unique identifier to anonymise the data prior to analysis. Data was processed and cleaned by removing non-valid responses, duplications, and any offensive material. All removed responses were collated and stored securely for reference. For further information on the results of the data cleaning and final number of responses and/or comments extracted from each evidence strand for analysis, see Annex D.

Summary statistics were produced in excel for quantitative responses to questions 1-3 of the public written consultation. 

Thematic analysis of text-based responses was completed manually within excel. Thematic analysis is the process of identifying patterns of meaning within qualitative data and categorising them into themes through coding. During coding, passages of text were extracted from the evidence and assigned codes which represented themes of interest. Each piece of evidence was analysed and coded through the lens of our review question “Do the National Outcomes still reflect the kind of Scotland we want to see?” with a focus on the following four areas:

1. Possible changes to the current set of 11 National Outcomes 
2. Data gaps and new topics that should be included in the National Outcomes
3. Changes to the wider National Performance Framework or set of National Indicators
4. The main obstacles and barriers in the further implementation of the National Outcomes

For ease of further analysis and distribution across the NPF unit, a coding hierarchy was developed to assist with categorisation. Initial codes were focussed on themes relevant to the above four questions. Thematic analysis is an iterative process, therefore these codes changed over time as we explored the data, found new patterns, and condensed together similar themes. The final themes and sub-themes were split into three categories for distribution and further use: 

1. National Outcomes and Indicators 
2. Implementation Gap 
3. Other 

The structure of the final coding hierarchy, including examples, has been set out in Annex E. 

Although some themes share the same name as the current outcomes, it should be noted that not all comments (and therefore coded themes and sub-themes) were specific requests to change the National Outcomes. Rather themes were developed around the current outcomes for ease of considering topics that were important to respondents and organisations for inclusion in updated outcomes.

Table 1. An example of coded text from thematic analysis of the evidence 

*Evidence extracted from the Doing Politics Differently: Citizens’ Assembly of Scotland (2020) report as part of the Desk-based Evidence Strand

In order to quality assure the manual coding process, a 10% sample of the responses were coded into themes by a different person in the team and results compared. Any discrepancies were addressed and resolved by E&R consensus. A double check was completed to ensure that every free-text response had at least one code attached to it so every view was accounted for. 

High-level themes and sub-themes were summarised including the number of associated comments. Quotes have been extracted, where permission was given in the Respondent Information Form (Annex F), to demonstrate the length and breadth of the views that were analysed. 

Results

Written Consultation Questions 1-3 Summary Statistics

Summary statistics were produced for the quantitative responses to questions 1-3 of the public written consultation (Table 2 and Table 3). Knowledge of the NPF varied with responses across all options on the 1-5 scale. Most respondents had at least some knowledge of the NPF with only 15% of respondents stating they knew nothing at all about the NPF.

Table 2. Responses to Q1 “How much would you say you know about Scotland’s National Performance Framework?” as a percentage of respondents 

When asked if the current set of National Outcomes fully described the kind of Scotland they wanted to see, two-thirds (66%) responded positively. The percentage of respondents who agreed for individual Outcomes was much higher (83-93%). This suggests that although generally content with the individual outcomes, there were themes missing from the National Outcomes set overall that respondents wanted to see. The highest level of agreement was with the Children and Young People and Environment Outcomes, with 93% of respondents answering “Yes” when asked if the individual outcomes described the kind of Scotland they want to see. The lowest level of agreement was with the Economy Outcome (83%), closely followed by the Poverty and Health Outcomes (both 85%). 

Table 3. Percentage of respondents who said “Yes” to Q2 “Do the current set of National Outcomes fully describe the kind of Scotland you want to see?” and Q3 “Does each individual National Outcome describe the kind of Scotland you want to live in?”

High-level Themes

Responses analysed were diverse and covered a wide variety of themes including comments related to all the current outcomes as well as new topics for future consideration. In addition, some comments were very specific requests to change the language of an outcome whilst others were general statements and aspirations for Scotland as a nation. The key themes and sub-themes have been discussed below. Detailed tables of the emerging themes, sub-themes and associated number of comments can be found in Annex G.

Communities

The most comments were on the theme of Communities (440) with 30 additional requests to add new communities-related indicators and 24 requests to change or remove current community-related indicators. 

The sub-themes within the Communities theme were diverse, with a large focus on local actions rather than specific changes to the National Outcome. This included better infrastructure maintenance of community spaces and facilities as well as improved access to amenities and services, including sports and leisure facilities to increase participation. Other key sub-themes included calls to improve community engagement as well as community resilience, safety, and connectedness. 

“Stromness high street becomes more vibrant, with clean and well maintained pavements, lanes and closes” – Stromness Local Place Plan

“No leisure facilities for families/locals” – Mercat Cross & City Centre Local Community Plan

“we feel that there is also a gap around community activism, with an outcome that encourages volunteering, activism, community development and involvement in local democracy such as community councils, participatory budgeting and so on” – Community Enterprise

“It would be worth bringing in the notion of sustainable communities, resilient communities” – South Lanarkshire Council

“People of Greenock South and South West feel safe to go out at night” - Greenock South and South West Locality Plan

“There is the potential to add to the existing outcomes about being “digitally connected”, as this will key across all sectors as we look to the future” – Scottish Tourism Alliance

“explicit reference to connectedness, reflecting the centrality of social connections to individual and community wellbeing” – Mental Health Foundation

Over 300 of these comments were sourced from the desk-based research. Due to the large volume of community action plans analysed through this evidence strand, the focus on communities and actions to improve the local area is unsurprising. The community-themed comments accounted for almost a quarter (24%) of the National Outcomes and Indicators evidence extracted from the desk-based research compared to 1-6% for the other evidence strands. 

Education

There were 361 comments related to the theme of Education with 14 additional requests to add new education-related indicators and 12 requests to change or remove current education-related indicators. 

The sub-themes within this were diverse, including calls to action to improve inclusion and diversity within education settings, develop high-quality education professionals and shift focus to practical and lifelong skills development, including digital skills. 

“We heard participants call for a shift in from considering equality as everyone being treated the same to an appreciation that equitable education requires also valuing of the diversity of children, young people, and adults and may involve different approaches and supports to meet their individual and group needs.” - All Learners in Scotland Matter: national discussion on education final report

“Teachers in Scotland should be skilled, well-educated, qualified, experienced, and have time for ongoing training and professional development. We heard repeatedly about the importance of better preparation, training and support for teachers and professional staff, at all levels.” - All Learners in Scotland Matter: national discussion on education final report

“National outcomes for education seem very youth-focused - can there be a stronger emphasis on community/lifelong learning?” - Learning for Sustainability Scotland

“We should ensure that lifelong learning is accessible to all with an equal focus on learning new skills to enable all to contribute to society.” - Fossoway and District Community Council

270 of these comments came from the desk-based research, with the large majority sourced from analysis of a single data source: the 2023 National Discussion on the Future of Scottish Education. The education-themed comments accounted for almost a fifth (19%) of the National Outcomes and Indicators evidence extracted from the desk-based research compared to 1-7% for the other evidence strands. 

Care

There were 266 comments related to the theme of Care with 90 additional requests to add new care-related indicators. Oxfam Scotland led a campaign, A Scotland that Cares, backed by over 50 organisations that is calling for a new, dedicated National Outcome focused on care and carers in Scotland to end the invisibility of care in the National Outcomes, alongside the introduction of associated indicators. Several of the signatory organisations, including Oxfam Scotland, Alzheimer Scotland and University of the West of Scotland, provided evidence to support this through the call for evidence, with Oxfam Scotland also submitting a call for evidence response on behalf of the campaign coalition and providing representation at the Care expert stakeholder meeting.

The Care sub-themes were focussed on valuing and supporting carers and those experiencing care as well as the need to recognise the contribution of care to the economy and the universal need and use of social care across people’s lifetime. 

“Over time, a dedicated National Outcome on care will help to recognise care, redistribute care work, reduce the care burden for the individual, better reward paid care work, and allow care workers better representation.” – University of the West of Scotland

“I would like to see some acknowledgement about the importance of social care helping people to thrive and live a fulfilling life.” – Anonymous individual

“Everyone needs care at some point in this life: as a child, in older age, or due to ill health or disability. The call for a National Outcome on Care encompasses all forms of care. In addition to the significant positive impact of care on people receiving it, helping support them to live their best lives and realise their human rights to health and equal participation in society, care is a benefit to everyone” - A Scotland that Cares (Campaign Coalition)

There was an even spread of care-themed comments across three of the evidence strands, accounting for 8% of evidence from the written consultation, call for evidence and expert stakeholder meetings. The exception to this was the desk-based research where Care featured in fewer than 5 comments. 

Economy

There were 255 comments related to the theme of Economy with 22 additional requests to add new economy-related indicators and 33 requests to change current economy-related indicators. 

The key sub-themes included shifting focus of the National Outcome towards a wellbeing economy that is sustainable, deprioritising economic growth, becoming more locally focussed in supporting independent local businesses and with better, fairer use of resources. 

“There should be mention of a wellbeing economy. Ideally we should move away from the tradition measures of a "successful" economy and look to the health and wellbeing of the whole society, most notably the most vulnerable in society, as a measure of progress and success.” – Anonymous individual

“Revise the “Economy” Outcome to make explicit that our economy should serve the purpose of delivering good lives for all people and protect the health of our planet in a way that prioritises collaboration and care over competition and GDP growth.” – Wellbeing Economy Alliance Scotland

“Strive to be as self-sufficient as possible, with a competitive Scots circular economy that meets everyone’s needs in a fair way” – Scotland’s Climate Assembly

There was an even spread of economy-themed comments across all of the evidence strands, suggesting that these themes are important to both individual and organisational respondents through the public and stakeholder engagements. 

Human Rights and Equalities/Reducing Inequalities

There were 240 comments related to the theme of Human Rights and 32 requests to add a related but dedicated equalities/reducing inequalities outcome. As the sub-themes within these two themes were similar, the evidence has been looked at here in combination. There were also 40 requests to add new rights- or equality- related indicators and 13 requests to change or remove current rights-related indicators. 

The sub-themes within these themes were diverse, with a large portion of comments related to democratic process improvements such as improving accountability, transparency, upholding justice and citizen involvement in decision-making. 

“Scotland should be a country where the people and government communicate with each other honestly and respectfully; whilst working together with concise and factual information, based on openness and accountability.” - Doing Politics Differently: Citizens Assembly of Scotland

“I would like to live in a Scotland that has community-led political decision-making systems, including citizens' assemblies, in place, which allow the emergencies of climate and ecological collapse to be given the due attention and urgent action they require.” – Anonymous Individual

Other key sub-themes included improving equality, including gender, disability and race equality, and the upholding, respecting and strengthening of rights, including children’s rights. 

“the reference to freedom from discrimination represents a critical dimension of equality. However, it is only a partial view of equality, as it implies only a negative approach to equality by focusing on the avoidance of discrimination. While this is necessary, we must also recognise the national importance of proactively advancing equality by taking specific actions to improve outcomes for people sharing different protected characteristics.” – Equality and Human Rights Commission

“The current National Outcomes lack a gendered lens throughout.  A gendered lens allows policymakers to understand how those of different genders experience the world, and how this in turn, impacts policy in practice.  The current outcomes include vital areas of focus for Scotland, however, unless the Scottish Government attempts to understand how women and men interact with systems differently, these outcomes will never be met.  Women we have spoken with have explained that considering the gendered element of the world is key in working towards equity in Scotland and, would like to see this approach taken on a wider scale to tackle women’s oppression.” – Scottish Women’s Convention

[I would like to live in a Scotland that…] is a world leader in protecting and advocating for rights, where everyone is able to play an active role in shaping society and making decisions which affect their lives, and where resources are directed to those most marginalised and underprivileged in society.” – Anonymous individual

Almost half of the human rights and equality-related comments were sourced from the written consultation, accounting for over a fifth (22%) of the National Outcomes and Indicators evidence extracted from the written consultation compared to 4-6% for the other evidence strands. This suggests that these themes are important to the private individuals who responded through the public consultation.

Transport, Travel and Connectivity

There were 225 comments related to the theme of Transport, Travel and Connectivity with 18 additional requests to add new related indicators. A dedicated Transport, Travel and Connectivity theme was extracted due to calls for this to be added to the NPF as a distinct outcome however it is recognised that this theme is closely related to aspects of Communities.

As with Communities, there was a large focus on local actions within the sub-themes including improved infrastructure maintenance, better road safety and traffic calming measures and better public transport services.

“Carry out an audit of streets and paths to ensure there are good quality cycling paths and well-lit streets” – Denny, Dunipace and Dennyloanhead Community Action Plan

“No-car zone for non-residents - supported by better public transport (less car traffic is the future for cities)” - Mercat Cross & City Centre Local Community Plan

Other key sub-themes included better promotion of active travel as well as recognition of transport as an enabler due to it’s role in physical connectivity. 

“Priorities have changed with net zero and reducing car use so we should reflect that. The impact of transport is significant. The first step is probably to reduce travel. Most travel is over very short distances, could be replaced by means other than cars” – Transport Expert Stakeholder Meeting Notes

“Mobility is a vital enabler for disabled people to contribute to every aspect of Scotland’s social, cultural, and economic life. Our position being that transport is an enabler of human rights and must be ‘Available, Accessible and Affordable’. - Mobility and Access Committee for Scotland

Over two-thirds of the Transport, Travel and Connectivity comments were sourced from the desk-based research. Due to the large volume of community action plans analysed through this evidence strand, the focus on transport, travel and connectivity is unsurprising. These comments accounted for 10% of the National Outcomes and Indicators evidence extracted from the desk-based research compared to 0-6% for the other evidence strands. 

Housing and Homelessness

There were 210 comments related to the theme of Housing and Homelessness with 52 additional requests to add new related indicators. 

The sub-themes within this were diverse but key topics included the need for affordable housing, recognition of housing as an enabler to other outcomes and that access to adequate, accessible housing that meets needs is a human right. 

“Affordable housing: Develop housing for local people so that they don't have to move away.” - Stratherrick and Foyers Community Assembly

“Social justice cannot be achieved if people do not have a home they can thrive in. A whole system approach is required to address the growing housing emergency and fix the broken and biased housing system. This is why it is so important that housing is prioritised within the National Performance Framework. If urgent action is not taken, we will be unable to address the growing housing emergency and that will make it much more difficult to achieve the National Outcomes particularly those relating to poverty, health, education and employment.” – Shelter Scotland 

“For there to be specific mention of access to housing and for every one of Scotland's residents to have somewhere safe and secure to live.” – Anonymous Individual

There was a spread of Housing and Homelessness comments across all the evidence strands (2-9%), suggesting that these themes are important to both individual and organisation respondents through both the public and stakeholder engagements. It should be noted that 40% of comments originated from one expert stakeholder meeting which was specifically focussed on the topic of Housing and Homelessness, with representatives in attendance from various organisations including Crisis, Homeless Network Scotland, Shelter Scotland, Public Health Scotland and the Association of Local Authority Chief Housing Officers.

Health

There were 187 comments related to the theme of Health with 31 additional requests for a dedicated mental health and wellbeing outcome, 35 requests to add new health-related indicators and 28 requests to change or remove current health-related indicators. 

The sub-themes within this were diverse but key topics included the need for parity between physical and mental health within the National Outcomes, higher prioritisation and better support for mental health care, a high-quality healthcare system delivered through a fully supported and resourced NHS and recognition that healthcare is a lifelong right. 

“we would propose new wording to reflect the parity and inter-related nature of mental and physical health and to centre the importance of reducing health inequalities.” – Mental Health Foundation

“I want to see a separate national outcome for mental health. A lot of the ills in Society, e.g., substance abuse, alcoholism, criminality, gambling addiction, poverty, domestic abuse, etc have poor mental health. By improving diagnosis, making treatment patient centric, providing early intervention, educating the public on preventative techniques and supporting everyone who needs it, we can make savings in criminal justice, in physical health, improve educational attainment, improve employment outcomes, and reduce economic inactivity.” – Disability Equality Scotland

“I think healthy and active is fine, but it leaves out that a large proportion of health is access to free and timely healthcare, and also feels like it focuses on physical health and doesn't touch on mental health.” – Anonymous Individual

“have properly resourced and managed health and social care services, built around individuals and communities to achieve good health and wellbeing for all.” - Doing Politics Differently: Citizens Assembly of Scotland

There was a spread of Health comments across all the evidence strands (2-9%), suggesting that these themes are important to both individual and organisation respondents through the public and stakeholder engagements. 

Environment and Climate Action

There were 168 comments related to the theme of Environment with 67 additional requests for a dedicated climate change outcome. As the sub-themes within these two themes were related, the evidence has been looked at here in combination. There were also 87 requests to add new environment-related indicators and 43 requests to change or remove current environment-related indicators. 

The key sub-themes included recognising the dependence on the environment to deliver other outcomes, the need to prioritise combatting climate change, increasing sustainability and achieving net zero targets, actively protecting the environment, reducing waste and improving biodiversity.

“The natural environment is fundamental to wellbeing and is in a state of crisis, and these facts are inadequately captured by the National Outcomes at present” - Environment and Economy Leaders Group Engagement Session Notes

“We consider that the existing outcomes cover the right areas, but the wording of many needs to be refreshed to better reflect the urgency of the climate and nature related emergencies we are facing.” - NatureScot

“We believe there is a fundamental need to focus the country’s collective mindset on the climate emergency and the requirement for immediate action” – Scotland’s Climate Assembly

“we value, enjoy and actively protect our environment adapting and mitigating against climate change” – Anonymous Individual 

There was a spread of Environment themed comments across all of the evidence strands (4-6%), suggesting that these themes are important to both individual and organisation respondents through the public and stakeholder engagements. 

National Indicator Set

There were 599 requests to add new indicators across most themes with an additional 219 requests to change or remove current indicators. Key indicator sub-themes include adding new Care indicators (90 requests), new Environment indicators (59 requests), new Housing and Homelessness indicators (52 requests), new Fair Work and Business indicators (42 requests) and changing current Environment indicators (42 requests).

The majority (73%) of indicator-specific requests came from the call for evidence and were requested alongside proposed changes to the outcomes. This was the only evidence strand where guidance included a prompt requesting feedback on the indicator set which could account for this evidence strands dominance within the individual indicator-related themes.

Indicator-specific requests were very diverse and generally reflected the individual interests of the responding organisations. For example the Glasgow City Food Plan: Food Education Working Group suggested expanding the scope of over 20 indicators to include food-related data across seven current outcomes (including Environment, Culture, Communities and Economy) alongside their request to have the importance of food better reflected across the whole of the NPF. 

“food can be seen as being at the nexus of our most pressing national and local concerns. It is instrumental in achieving social and environmental justice and in extending sustainability as a cultural value from nature to business, education and communities. Rather than inserting food into any of the indicators or goals of the NPF, we think a holistic approach to food is of paramount importance and propose a holistic inclusion of food across the Framework.” - Glasgow City Food Plan: Food Education Working Group

Similarly, Close the Gap requested adding/changing over 20 indicators to improve gender mainstreaming within the NPF and track progress in reducing gender inequalities across Scotland.

“Close the Gap’s primary critique of the National Performance Framework (NPF) is that it is not well-gendered.” – Close the Gap

In addition to the indicator-specific requests, there were 181 requests on the indicator set overall and 81 requests for better and more data disaggregation. Key sub-themes within the indicator set comments included requests to change how we measure progress, further develop the indicator set and fill data gaps, consider the use of different data types such as lived experience and improve the quality of the underlying data. As with the indicator-specific requests, the majority (92%) of indicator set comments came from stakeholders through the call for evidence or expert stakeholder meetings.

“The Scottish Government must prioritise ensuring indicators are fully developed and consult with relevant civil society groups and stakeholders to ensure data and indicator selection are fit for purpose.” – Oxfam Scotland

“there is a need to change the way that we gather and understand data. This includes going beyond simply evaluating public services to understanding the outcomes and experiences for citizens and communities. It also includes paying greater attention to dimensions of inequality, recognising that population level data sets often ignore the experiences of marginalised groups. Both of these drivers would require blending the existing national indicators with a much greater emphasis on coproduced and qualitative evidence in order to truly understand progress towards delivering National Outcomes.” – Carnegie UK

“The indicators would benefit from being updated and should be based on a thorough review and analysis of their effectiveness, relevance, and alignment with the desired outcomes and should reflect current priorities and challenges such as the impacts of Covid-19, the cost-of-living crisis and Brexit.” – Culture Counts

Indicator themes accounted for 44% of the National Outcomes and Indicators evidence extracted from the call for evidence and 33% of the expert stakeholder meetings evidence compared to 1-3% for the desk-based research and written consultation evidence strands, suggesting that the data and evidence that tracks progress towards the outcomes is an important aspect of the NPF for stakeholders.

Implementation Gap

There were 874 comments related to the Implementation Gap, focussed around five key themes of Policy (171 comments), Delivery (166 comments), Funding (114 comments), Legislation (98) and Accountability (98). 

The key sub-themes within these included the need for better policy coherence, including better alignment of policies and strategies to simplify the complex reporting landscape, embedding the NPF in practice as a driver of change to close the gap between ambitions and delivery, better delivery of the Sustainable Development Goals (SDGs) through the NPF, improved funding models, better use of the NPF in informing budget decisions, support for the development of a Wellbeing and Sustainable Development Bill as well as calls for increased accountability mechanisms including clarifying roles and responsibilities, better scrutiny and improved government transparency of actions taken and progress towards outcomes.

“one significant barrier to implementation is the lack of guidance that clarifies how public bodies should use the National Outcomes to guide decision making, in the context of a myriad of other policy frameworks. While there are some areas where the National Outcomes are more visibly embedded, there are many places where other statutory duties or non-legislative frameworks are seen to take precedence.” – Carnegie UK

“the NPF needs to be used more systematically and explicitly to influence decision making if it is to be the golden thread from which all other policies and strategies connect to delivery on the ground.” - Finance and Public Administration Committee (FPAC) inquiry report

“Ensure regular and detailed progress reporting on implementing the SDGs (via the National Outcomes or directly to the UN) in the spirit of the UN’s commitment to localising the implementation of the SDGs.” – SDG Network Scotland

“In terms of giving greater profile to the NPF, could new initiatives have to set out which Outcomes they are delivering on. There’s a disconnect between spending and delivery of the National Outcomes.” – Transport Expert Stakeholder Meeting Notes

“Strengthen duties to promote National Outcomes through delivering the promised Wellbeing & Sustainable Development Bill and provide clear guidance for implementation.” – Wellbeing Economy Alliance Scotland

“looking to the role of existing public audit and scrutiny bodies to support improvement and hold bodies to account for working collaboratively towards the outcomes and to adopt preventative actions and long-term perspectives.” – Public Health Scotland

These implementation gap-related comments were largely driven by requests from stakeholders through the call for evidence which accounted for over 50% of comments, suggesting that these themes are important to our stakeholders. This may also in part be a product of the data collection method for this evidence strand, where guidance included a prompt to think about what the main obstacles and barriers are in the implementation of the National Outcomes. However, a further quarter of comments were sourced from the expert stakeholder meeting evidence with only 3% of comments coming from the public written consultation responses. 

Other Themes

Of the 345 other comments, 137 were positive comments about the NPF, including that it is a useful high-level vision with good comprehensive coverage of key issues. Positive feedback was received through every evidence strand however the majority (64%) came from the expert stakeholder meetings. 

“Overall aspirational and positive” – Open Expert Stakeholder Meeting Notes

“We believe the NPF provides a key message that wellbeing is at the heart of Scotland’s collective ambitions.” – Scottish Futures Trust

“We recognise that there is a lot to celebrate and praise about the NPF, including its relevance in tracking progress in a modern society, it’s diversity of important outcomes, and its relevance within Scottish Government.” - BiGGAR Economics

“I think that these National Outcomes apply to all societies and to all parts of the United Kingdom and the world. I think that they represent things that we should all aspire to.” – Anonymous Individual 

The most common other themes were requests to improve the format, including accessibility and visual presentation, changes to the website content and functionality, such as better visibility of disaggregated data and simplified site navigation, requests to change the purpose wording to focus on wellbeing and sustainability as well as calls to change the name of the NPF to better reflect that it is a wellbeing framework. 

“I think that a better representation of a wellbeing economy would be “to focus on creating a more successful country with opportunities for all of Scotland to flourish through increased wellbeing, and a sustainable and inclusive economy”” – Anonymous Individual 

“First and foremost, this review of the National Outcomes is an opportunity to enhance and clarify the overarching purpose of the NPF as Scotland’s ‘National Wellbeing Framework’. This should build on the welcome addition during the last review in 2018 of the ‘purpose’ statements, and specifically the aim to: “increase the wellbeing of people living in Scotland”. The name of the NPF should be changed to focus on this. Critically, there should also be meaningful differentiation between this core purpose and the means of achieving it. Specifically, the current aim “to create sustainable and inclusive growth” is a means to support national wellbeing, not an end in itself, and it therefore has no place in the purpose statement.” – Oxfam Scotland

“Changing the title from the National Performance Framework to one that better reflects the ambition of the approach would provide a clearer articulation of what the National Outcomes should do for the people of Scotland and have greater relevance to citizens.” – Carnegie UK

Comments related to these themes were received through every evidence strand however over 90% came from stakeholders through either the call for evidence or expert stakeholder meetings.

Conclusion

The statutory Review of the National Outcomes seeks to answer the question “Do the National Outcomes still reflect the kind of Scotland we want to see?” and to collect evidence on the implementation gap of the National Outcomes across Scotland.

Analysis of evidence collected through four evidence strands showed that views were diverse, with over 5,000 comments categorised into over 100 themes and 800 sub-themes. Although the NPF generally provides a useful high-level vision with good comprehensive coverage of key issues, there are gaps and areas which could be strengthened to better reflect the kind of Scotland we want to see. Key themes which are underrepresented within the current National Outcomes include: recognising the importance of care, good quality transport and housing, shifting focus towards a wellbeing economy, reducing inequalities, improving mental health, and prioritising climate action to combat climate change. 

Analysis of evidence on the implementation gap of the National Outcomes noted several common barriers including policy coherence, a complex reporting landscape, difficulties embedding the NPF in practice as a driver of changes, and dissatisfaction with current funding models. There was support for the development of a Wellbeing and Sustainable Development Bill as well as calls for increased accountability mechanisms including clarifying roles and responsibilities, better scrutiny and improved government transparency of actions taken and progress towards outcomes.

Once analysed, themes and frequency of occurrences, were distributed across the NPF unit:

• all the results were fed into the policy development of the new NPF National Outcomes
• the National Outcomes and Indicators thematic analysis results will feed into the development of the updated National Indicator Set in 2024 to support the new National Outcomes
• the Implementation Gap category of themes, and associated evidence, will feed into the development of the implementation plan. These results were also passed onto the legislation team to continue work on the WSD Bill.
• the Other category of themes will be looked at on an individual basis and taken forward for consideration with relevant members of the NPF unit.