Intelligibility in English as a lingua franca – The interpreters’ perspective

Inovative Logic Crosstrainer II v4.5.7 English by EViDENCE

Inovative Logic Crosstrainer II v4.5.7 English by EViDENCE

Figure 3-2-1 Project Evaluation Methods and Their Application to. Evaluation Studies. Conclusion.. Utilization of the logframe logic. v 2 Five Basic Parts of the Organization. 18. V 3 The Organization as a System of Flows. 35 r P~RT" - THE DESIGN PARAMETERS. 6S.,/4 Design of Positions. Competency assessment and certification;. 2. Registration and delivery of training programs; and. 3. Development of curriculum and assessment instruments. Each.

Intelligibility in English as a lingua franca – The interpreters’ perspective

This article aims at examining the topic of ELF TotalSpaces 2.9.9 Cracked for macOS Free Download 2021 from the interpreters’ perspective. Therefore, the focus is put on listener factors affecting intelligibility in settings typical for interpreting i.e. monologic settings. Data from various intelligibility studies are compared with results from a study that tested an ELF user’s intelligibility in a conference-like ELF setting and examined the influence of listener variables such as background knowledge, familiarity with ELF use or proficiency in English. In this study, an Italian speaker gave an impromptu speech in English to participants who subsequently were asked to answer written questions on the topic. The results showed that listeners with more experience in ELF settings reached the highest score in the test, while participants with specialist knowledge were unable to profit from it. The participants’ English language skills played a rather subordinate role. The findings of this study may prove useful for considerations in interpreter training and can contribute to the development of concrete, evidence-based training methods for interpreters in the interpreting sub-skill of comprehension.

Das Thema Verständlichkeit von ELF-SprecherInnen wird in diesem Artikel aus dem Blickwinkel der Dolmetschwissenschaft beleuchtet. Daher liegt der Fokus vor allem auf ZuhörerInnenfaktoren, die Verständlichkeit in typischen monologischen Dolmetschsettings beeinflussen. Daten aus verschiedenen Verständlichkeitsstudien werden mit Ergebnissen aus einer Studie in Zusammenhang gesetzt, die die Verständlichkeit eines ELF-Nutzers in einer konferenzähnlichen Situation testete und ZuhörerInnenvariablen wie Hintergrundwissen, Vertrautheit mit ELF-Kontexten oder Englischkenntnisse überprüfte. Dabei hielt ein italienischer Redner einen frei gesprochenen Vortrag auf Englisch, zu dem den TeilnehmerInnen im Anschluss schriftliche Verständnisfragen gestellt wurden. Bei der Auswertung der Antworten zeigte sich, dass ZuhörerInnen vor allem von viel Erfahrung mit Kommunikation in ELF-Settings profitierten. Die TeilnehmerInnen mit Fachwissen im Bereich Inovative Logic Crosstrainer II v4.5.7 English by EViDENCE Vortrags konnten hingegen keinen Vorteil daraus schlagen. Die Englischkenntnisse der ZuhörerInnen spielten eine eher untergeordnete Rolle. Diese Erkenntnisse könnten in der Dolmetschlehre genutzt werden und dazu beitragen, konkrete, evidenzbasierte Ausbildungsmaβnahmen für DolmetscherInnen in der Teilfertigkeit Verstehen zu erarbeiten.

Keywords: ELF intelligibility; variables affecting intelligibility; familiarity with ELF use; background knowledge; listening comprehension

Schlagworte: ELF-Verständlichkeit; Einflussfaktoren; Vertrautheit mit ELF; Hintergrundwissen; Hörverständnis

1 Introduction

The increasing use of English as a lingua franca (ELF) is a development that has radically changed international communication and has also had a major impact on one of the different forms of transcultural communication: interpreting.

Even though the volume of research in the field of ITELF (interpreting, translation and English as a lingua franca) is still relatively low, several surveys and studies have shown that interpreters often struggle with source texts produced by ELF users and that such input can have adverse effects on interpreters’ output (for an overview, see Albl-Mikasa 2018).

Therefore, many interpreting scholars agree that interpreters will have to do better at adapting to changing circumstances and demonstrate their added value in ELF settings if their service is to coexist with lingua franca communication. Hence, practising and future interpreters must be specifically trained in the concrete skills necessary to provide high-quality interpretation in ELF settings, a demand already raised by interpreters as early as 1952 (Herbert) and even more so since the significant increase in the use of English as a lingua franca (Albl-Mikasa 2013; Huh 2017; Reithofer 2010). Nevertheless, apart from a study by Albl-Mikasa et al. (2017), interpreter training in this field has not gone beyond anecdotal evidence of coping strategies for interpreting ELF users or mere suggestions of what might work.

It is widely recognised that interpreting is SplashID Desktop (for Android) 4.6.0 crack serial keygen of various sub-skills, often divided into reception (listening and comprehension), transfer and production. Many training approaches spilt these part-tasks up even further and suggest to also practise them independently in a component-skills approach (for an overview, see Setton and Dawrant 2016: 60–63).

Without doubt, one of the biggest challenges and hence a key sub-skill required for interpreters in mediated ELF contexts is comprehension. These contexts are typically monologic settings where one speaker gives a talk and interaction is very limited – not only for the interpreters, but also for the audience. These unidirectional communication settings make it largely impossible to use common ELF strategies. Even though studies show that ELF users do apply pro-active strategies such as repetition or paraphrasing even in monologic settings (e.g. Björkman 2011), other essential ELF strategies such as negotiating or co-constructing the meaning, requests for clarification or repetitions are difficult to use by listeners in these types of encounters (Björkman 2011: 960) and virtually impossible for interpreters. If interpreters wish to enhance their comprehension sub-skill for assignments in ELF contexts, evidence-based knowledge about listener-related factors that allow better comprehension of ELF users is essential.

Therefore, this paper will explore the concept of ELF intelligibility from the interpreters’ point of view and review studies on listener factors affecting intelligibility.

Furthermore, it will present data from a study with a typical conference-type ELF setup characteristic for interpreting contexts with an expert ELF user speaking to a multilingual audience. The study was aimed at assessing the listener variables that affected comprehension by correlating the participants’ results in a comprehension test with background variables collected in the study – such as the listeners’ background knowledge, familiarity with use of English or proficiency in English. The results should help identify the factors that facilitate the interpreting part task of reception in an ELF context to then use this knowledge in specific ELF training for interpreters.

Since listening and comprehension are skills that are not unique to interpreting, this paper considers findings from all listening contexts comparable to mediated settings to be valuable for the training and continuous professional development of interpreters regarding the part-task of listening and comprehension.

2 Intelligibility in ELF contexts

Multiple definitions of the notion of intelligibility have been developed in the field of linguistics. A frequently used definition is the threefold distinction proposed by Smith (1992: 76), who differentiates between

  • intelligibility (recognition of words/utterances),

  • comprehensibility (understanding the meaning of a word/utterance [locutionary force]) and,

  • interpretability (understanding the meaning behind a word/utterance [illocutionary force]).

Smith also assumes that the three levels interact. One may ask, however, whether intelligibility, malwarebytes premium 3 and interpretability are required to ensure good interpreting performance. In interpreting studies, most scholars base their concept of comprehension on the well-established notion grounded in cognitive psychology that comprehension occurs on the micro and the macro levels: recognition and decoding of linguistic information Inovative Logic Crosstrainer II v4.5.7 English by EViDENCE combined with prior knowledge and setting-specific facts in order to transfer the speakers’ input in a given situation (Padilla and Bajo 2015). This would imply that the source text would have to be intelligible and comprehensible. In some instances – when interpreters merely transcode the message e.g. in lists or enumerations – intelligibility might suffice. Interpreters often claim to interpret what “the speaker actually wants to say” and thus refer to interpretability, but this is difficult to measure and has thus been largely disregarded in linguistics (Pickering 2006: 220).

In the following, I will explore in greater detail which factors have an influence on intelligibility, which I will use as the umbrella term for the three concepts and indicate when they are used in Smith’s sense.

2.1 Variables affecting the intelligibility of ELF users

This section will discuss relevant findings on ELF users’ intelligibility from different fields of linguistics, some also from outside the ELF paradigm who mainly looked at non-standard English.

While it has been acknowledged that understanding is co-constructed by speakers and listeners (Pickering 2006), research has focussed largely on speaker factors affecting intelligibility.

Among the most frequently studied topics are accent and phonology, but variables such as speakers’ lexis, syntax or grammar have also been examined. A broad range of approaches have been used to determine intelligibility: in the field of ELF research, many scholars analyse interactions to identify factors responsible for a breakdown of communication (e.g. Deterding 2013; Jenkins 2000). In some studies, the participants are asked to jointly complete a task using English as a means of communication (e.g. Kennedy 2017; Thir forthcoming). Other analysts use scales to assess listeners’ subjective understanding of L2 speakers (Anderson-Hsieh et al. 1992; Munro et al. 2006). Furthermore, word or sentence recognition tasks (Wang and Van Heuven 2014), transcriptions (Kennedy and Trofimovich 2008) or cloze tests (Smith and Rafiqzad 1979) have been employed. Another approach relies on comprehension tests that listeners take after hearing an utterance by an ELF user (Bulatović et al. 2019; Wilang and Teo 2012a); this is the method used in the study described in Chapter 3.

When looking at intelligibility variables of ELF users cited in the field of interpreting studies, accents have often been mentioned as a difficulty for interpreters (Cheung 2015). Some interpreters believe that non-standard phonology (Chang and Wu 2014) or prosody (Lin et al. 2013) and thus intelligibility in Smith’s sense are the main problems in ELF contexts.

Albl-Mikasa et al. (2017), however, found that the main problem trigger for interpreters was not ELF users’ accent by itself, but rather non-standard phonology combined with unorthodox wording and phrasing. Due to the “speakers’ constrained pragmatic fluency and restricted ability to express their key point in precise terms […] interpreters find it hard to (quickly) grasp the argumentative logic behind the phrases and expressions” (Albl-Mikasa et al. 2017: 232). This implies that, in Smith’s terms, not only intelligibility, but also comprehensibility (and interpretability) pose problems for interpreters.

While it is likely that the level of intelligibility of an utterance is determined more by the speaker than by the listener (Hansen Edwards et al. 2018: 554), knowledge about speaker factors is less relevant to interpreters, since they cannot control or influence a speaker’s output. Interpreting researchers’ main goal therefore is to find ways to help practising and future interpreters to better understand the ELF users they have to interpret.

Turning thus to listener variables affecting intelligibility in ELF contexts, research has shown that they include a plethora of factors. I will focus mainly on the ones examined in the study that is to be reported upon in this paper.

2.1.1 Background knowledge

The essential role of prior topic-specific knowledge in the comprehension process is widely recognised (McNamara et al. 2007). In ELF studies, the question of background knowledge is closely related to the notion that shared knowledge enables experts from a domain to communicate effectively with each other using ELF in their community of practice. It has been assumed that experts in the field of the study described in Section 3 – business communication – can overcome linguistic barriers thanks to their specialisation in the respective field (e.g. Kankaanranta and Louhiala Salminen 2018). Poncini (2004) found a culture of its own at business meetings, a groupness that evens out difficulties arising quickbooks pro 2004 2004 pro crack serial keygen the speakers’ different levels of English proficiency.

Turning to experimental findings from other relevant fields of linguistics, early studies (Gass and Varonis 1984; Smith 1992) also showed that prior knowledge increased the intelligibility and interpretability of ELF users.

Interpreters’ prior knowledge cannot be assumed to equal their clients’ expertise, since they are usually not part of the expert groups they work for. Therefore, interpreters try to acquire as much subject knowledge as possible during preparation for a specific task or for domains they regularly work in (Kalina 2005). In Jogos de Submarino de Graça para Baixar experimental study (Díaz Galaz et al. 2015), thorough preparation and thus acquisition of background knowledge was shown to lead to better quality in interpreting difficult technical segments. Scholars believe that preparation also helps when eyeBeam 1.5 crack serial keygen text difficulty arises from non-standard input (Albl-Mikasa 2013; Huh 2017; Reithofer 2018) and practising interpreters likewise see it as a potential coping strategy (Chang and Wu 2014).

2.1.2 Knowledge of ELF users’ first language

One recurring feature observed in ELF contexts is that speakers creatively apply elements of their first language (L1) also when speaking English (e.g. Guido 2012). Many studies have thus investigated whether intelligibility of ELF users is higher when speakers and listeners share an L1, i.e. whether a German listener understands a German speaker of English better than a Spanish or an American listener would.

In this respect, Bent and Bradlow (2003) proposed the concept of the matched interlanguage speech intelligibility benefit, an advantage resulting from the interlocutors’ common knowledge about consonant and vowel categories, emphasis patterns, intonation, etc. Another explanation for this hypothetical advantage could be that persons with the same L1 typically share a common culture in the sense of common knowledge about (linguistic) conventions and the like (Kachru 2008: 311) and thus understand each other better in an ELF context.

This advantage was confirmed Inovative Logic Crosstrainer II v4.5.7 English by EViDENCE various studies in the field of ELF and beyond with very different designs (Jenkins 2000: 57; Lee 2004; Smith 1992; Wang and Van Heuven 2015; Yule et al, Inovative Logic Crosstrainer II v4.5.7 English by EViDENCE. 1990). However, Hansen Edwards et al. (2018) found that it depends on the language proficiency of listeners whether a shared linguistic background is beneficial to intelligibility or not. Tauroza and Luk (1996) also failed to find what they call an own accent advantage.

In the field of interpreting, Chang and Wu (2014) as well as Katikos (2015) concluded that interpreters understand ELF users from a common L1 background best and find those with significantly different language backgrounds most difficult to interpret.

Furthermore, there is evidence for a benefit of knowing a speaker’s L1, that is, not having the same L1, but understanding the speaker’s L1 very well. Several interpreting studies (Kraus 2011; Kurz and Basel 2009) examined this phenomenon and all found that knowing an ELF user’s L1 well or having it as a working language immensely facilitates the interpreter’s task. In reference to Bent and Easeus Data Recovery Wizard Crack v14.2.1 Method:, Albl-Mikasa (2013: 105) called this the shared languages benefit.

2.1.3 Familiarity with use of English

Studies from various linguistic sub-disciplines on familiarity with non-standard English will be reviewed in this section. Although not all of them are based on the ELF paradigm, I believe their findings are nevertheless relevant for this paper.

Some linguists believe that non-L1 listeners generally understand other non-L1 speakers better than L1 listeners of the language that is being spoken do. They see the reason for this in the fact that non-L1 listeners are familiar with non-standard use of the language and know and recognise strategies employed by non-L1 speakers. Bent and Bradlow (2003) coined the term mismatched interlanguage speech benefit for this hypothetical phenomenon.

Several scholars looked at how familiarity with non-standard speech in general affects comprehension. It has been observed that ELF users are more intelligible for listeners with previous exposure to non-standard speech (Kennedy and Trofimovich 2008), be it through international experience (Hansen Edwards et al. 2018) or contact with ELF use at work, through (social) media or while travelling or living abroad (Wilang and Teo 2012b).

Regarding familiarity with a specific accent or rather similect (Mauranen 2018), several studies showed that it increases the ability to understand the similect (Derwing and Munro 1997; Munro and Derwing 1999; Wingstedt and Schulman 1987).

Smith and Rafiqzad (1979) suggested that this familiarity materialises already after a short time of exposure. Weil (2001) indicates a period of four days, after which even different speakers with a similar accent are more intelligible. Clarke (2004) found that this normalization can become apparent after only a few hours or even after only 1 min or two to four sentences of exposure to the same speaker.

Practising interpreters confirm this and claim they are able to adapt to a specific accent in a few minutes (Chang and Wu 2014: 176–177). Student interpreters reported similar effects (Huh 2017: 73).

Familiarizing interpreters with different similects is frequently mentioned as a potential training strategy for ELF settings. Scholars assume that frequency of exposure allows for the storage of the specific linguistic patterns in long-term memory, which then facilitates the interpreters’ listening effort (Huh 2017: 60).

Other linguistic studies (Clarke 2000; Munro et al. 2006), however, rebut the advantage of this type of familiarity or find only weak evidence of a link to intelligibility.

2.1.4 English proficiency of listeners

Another variable that some researchers believe affects the intelligibility of ELF users is the listeners’ proficiency in English. Listeners with higher proficiency in English are thought to have fewer problems understanding ELF users.

Smith (1992) showed that listeners’ language proficiency influences intelligibility of ELF users, but that this does not mean L1 listeners are best able to understand ELF users. Wilang and Teo (2012b) found ELF listeners’ exposure to English in education to correlate with comprehension of ELF users. In their experiment, listeners with graduate studies in English were better at understanding ELF users. Matsuura et al. (2017) also found listeners who were more proficient in English to be better at understanding ELF users, since they relied less on the phonological input and were able to use contextual clues.

Turning to ELF settings typical for interpreting assignments, variables such as background knowledge, familiarity with ELF contexts or proficiency in English can be analysed by using data from a study which will be described in the following chapter.

3 Study

The findings described below are part of a larger research project (Reithofer 2013, 2014) that compared the effect of ELF and interpreting. In what follows, I will only report the part regarding ELF.

Unlike most ELF studies with dialogic settings, this study sought to create a context typical of the ones in which interpreters frequently encounter ELF users: a monologic talk by an expert speaker addressing a large (expert) audience with no or little room for interaction. Therefore, a conference-like setting was created where participants with various L1 were asked to listen to a conference-type speech by a proficient ELF user. Afterwards the listeners completed a comprehension test on the content of the talk.

3.1 Research question

As mentioned above, interpreters have no influence on the communicative resources of the ELF users they interpret. They can only maximise their level of understanding of the speakers’ input. In order to do this, interpreters and interpreting trainers need to be aware of the listener factors that facilitate comprehension of input by ELF users. In the following, the variables that affected comprehension in a conference-type setting will be assessed by correlating the participants’ results in the comprehension test with background variables collected in the study to find listener factors that increase comprehension.

3.2 Speaker

It has been acknowledged that intelligibility of ELF users can vary immensely (i. a. Wang and Van Heuven 2014). Since no single user can be regarded as the typical ELF user, it was decided to choose a realistic representative of presenters at conferences: a non-native speaker of English who frequently communicates Kontakt MAC License Key Archives high-level ELF settings. After video recordings with several academics at a prestigious Italian university, the author’s research team and international interpreting experts chose an Italian professor with extensive experience of lecturing in English – in his university courses and at international academic conferences. He had been recorded giving an Inovative Logic Crosstrainer II v4.5.7 English by EViDENCE speech of 15′24′′ without any notes or script on one of his main research topics in the field of marketing. The talk included typical features of authentic lecture discourse and spontaneous speech such as false starts, repetitions, redundancies and rhetorical questions.

The topic turned out to be highly innovative as confirmed in a pre-test with subject-matter experts.

The speaker’s accent was rated by 46 experts – interpreters and ELF scholars – on a 7-point interval scale from one (no perceptible foreign accent) to 7 (very strong foreign accent). Eight raters were L1 speakers of English, 38 were non-native speakers of English with various European L1. His accent was given a mean rating of 5.48 (SD = 1.1) and can thus be described as slightly stronger than average. The raters, however, also mentioned that he was very intelligible and had a high level of proficiency in English. Nearly all raters (93.5 %) Category Archives: Photo Editor him as an Italian speaker of English.

3.3 Questionnaire

The questionnaire administered to the participants consisted of two parts: one with listening comprehension items and the other with background questions on the participants and subjective assessments of various parameters. The listening comprehension test comprised multiple choice (MC) and (half-)open-ended questions that were based on the wording of the speech. Lexical overlap was explicitly desired to increase the possibility of recognition of information and to lower the need for recall, Inovative Logic Crosstrainer II v4.5.7 English by EViDENCE, which would have tested the participants’ Inovative Logic Crosstrainer II v4.5.7 English by EViDENCE capacity rather than comprehension (Berne 1993: 24). It is true that when information is queried directly after listening, not only the intelligibility of lectures is tested, but also the memory and retention of the test subjects. Since the process of understanding cannot be observed, Inovative Logic Crosstrainer II v4.5.7 English by EViDENCE, the test subjects had to be given some kind of task. The participants were comparable regarding their age, education and study situation. They were parallelized according to their overall grades in order to distribute Inovative Logic Crosstrainer II v4.5.7 English by EViDENCE as equally as possible, which was intended to ensure that the Inovative Logic Crosstrainer II v4.5.7 English by EViDENCE of the factor memory was kept as small as possible.

Eight out of 11 items were MC questions. The sequence of correct answers and distractors was changed from item to item in order to avoid format effects.

The three semi-open questions were phrased in such a way as to make the correct answers clear from the beginning (e.g. “Name the speaker’s three concrete examples for customers with a fragmented identity”). They were thus not fully open-ended questions, but, compared to closed questions, had the advantage that the right answer could not be guessed. A calculation of the level of difficulty of all items showed that they were not the most difficult and thus did not distort the results. All items were of medium difficulty, i.e. between 0.2 and 0.8, where 0 is the most difficult and one the least difficult.

The questionnaire was tested beforehand in cognitive interviews with a small number of business experts, as recommended by Willis (2005), to evaluate its suitability and intelligibility.

3.4 Participants

In this paper, we will focus on 67 participants out of 139 in the original study, since those 67 listened to the ELF user (while the others listened to the interpreter). We will disregard the 72 subjects who listened to the interpretation not relevant for this paper.

Among those listening to the original speaker were 42 business students at two Austrian universities of applied sciences (Fachhochschulen, referred to as FH1 and FH2) who have to pass highly competitive admission tests, Inovative Logic Crosstrainer II v4.5.7 English by EViDENCE. These specialist students can be considered a realistic target audience for the presentation by the university lecturer. 25 participants were interpreting students at the Vienna University’s Centre for Translation Studies (CTS) and thus non-experts in the field of marketing. All of the participants had L1s other than English (cf. Table 1).

Table 1:

Participants’ L1.


The participants Inovative Logic Crosstrainer II v4.5.7 English by EViDENCE comparable in terms of age (cf. Table 2).

Table 2:

Participants’ age and gender.

Number of participants271525
Mean age20.725.123.5
Age range19–2520–3322–27

As mentioned before, listening and comprehension are part-tasks of, but not skills unique to interpreting. Thus, findings from all types of listeners in ELF contexts that are comparable to mediated settings Inovative Logic Crosstrainer II v4.5.7 English by EViDENCE relevant for gaining Inovative Logic Crosstrainer II v4.5.7 English by EViDENCE insights into the variables affecting intelligibility in ELF contexts that could be used in the training for the interpreting sub-skill of reception.

3.5 Procedure

The conference simulation for the FH1 and CTS groups took place in a lecture hall at the Centre for Translation Studies. The questionnaire was handed out to the participants as soon as they entered the room, but they were told not to read it until instructed to do so. The procedure was then explained in detail. During the talk, students saw the original speaker on a large video screen. After the speech they were asked to complete the questionnaire.

The test with the FH2 group took place on the premises of that institution, with comparable equipment and facilities, and followed the same procedure.

3.6 Results

The effect of the variables of subject-matter knowledge, familiarity with ELF use or the speaker’s similect and language proficiency on intelligibility was investigated on the basis of inter-group comparisons as well as correlations between the score in the comprehension test and biography-related listener variables.

3.6.1 Background knowledge

Background knowledge or prior knowledge of the topic also referred to as subject-matter expertise has frequently been cited as a facilitating factor for ELF intelligibility (see Section 2.1.1). The business students were expected to share relevant knowledge with the speaker and thus understand him better than the interpreting students.

Apart from their expertise resulting from their field of study, FH1 and FH2 students also considered themselves to be more familiar with the subject of the speech than CTS students. In their responses to the question “How familiar were you with the topics discussed in the speech? (from 1 = very familiar to 7 = not familiar at all)”, there was a two points difference between the median of CTS and FH1 students and a one point difference between the CTS and the FH2 groups (cf. Table 3).

Table 3:

Familiarity with speech topic.


Nevertheless, the results of the comprehension test for the three groups showed that the non-experts from CTS had reached the highest score at 8.3, compared to 8.1 by FH and 5.7 by FH2 (cf. Table 4). Thus, even though FH1 and FH2 students participated in a course in the domain of the speech and claimed to be more familiar with the subject, the interpreting students achieved better results in the comprehension test.

Table 4:

Comprehension scores.


3.6.2 Familiarity with ELF use

Several studies indicate that listeners who have more experience in ELF settings and with ELF users of any linguacultural background also find them more intelligible (see Section 2.1.3).

Therefore, in our study, the participants were asked “How often do you speak English with persons who use English as a foreign language?”.[1]

The answers (very often, often, sometimes, rarely, never) were grouped into two categories: participants with more contact with ELF users (very often, often), and those with less (sometimes, rarely).

In all test groups, respondents in the “more experience” category achieved better test results: in FH1 by 2.6 points, in FH2 by 2.1 points and in CTS by one point (cf. Table 5). A t-test showed a significant difference in the FH1 scores for less contact (M = 6.82, SD = 2.4) and more contact (M = 9.43, SD = 3.6); t (25) = 2.24, p = 0.035. Because of the small number of participants in FH2, no t-test was conducted, but the 2-point difference is a highly meaningful descriptive outcome. Hence, the participants who spoke with ELF users (very) often also understood the Italian speaker better.

Table 5:

Test score in relation to ELF contact.


3.6.3 Familiarity with the speaker’s ELF similect

Another variable that, according to several studies, influences the intelligibility of ELF Nuclear Coffee VideoGet Key Features: is the listeners’ familiarity of the speakers’ specific accent or similect, in our case Italian English, Inovative Logic Crosstrainer II v4.5.7 English by EViDENCE. For this reason, respondents were asked to indicate (on a 5-point ordinal scale) how often they had heard Italians speak English.

In the FH1 group, 19 participants reported prior experience with Italian users of English. Surprisingly, the participants who stated that they had never heard an Italian speak English achieved the highest test score. This is in stark contrast to the findings discussed in Section 2.1.2. On the other hand, for those with experience, the test score increased with more experience. The results are, therefore, contradictory. It should be noted, however, that the group sizes were very small, and no one had chosen the answer option “very often”.

The FH2 and CTS groups did not show a clear picture either. No linear correlation was found in scatter plots. Due to the very small group sizes, no inferential statistics were carried out.

Thus, in this study, familiarity with the specific accent could not be confirmed as a factor positively influencing intelligibility.

3.6.4 Listeners’ L1

Overall, the number of participants grouped by their L1 is too small for general statements or statistical calculations about the influence of L1. They will be presented here to show a certain trend, Inovative Logic Crosstrainer II v4.5.7 English by EViDENCE, but cannot necessarily be extrapolated.

Almost all students in FH1 and FH2 indicated German as their L1. In the CTS group, five test participants shared the first language of the speaker – Italian. However, they achieved a lower test score than Multimedia Archives - Page 2 of 6 other listeners. The shared languages benefit therefore did not materialise (cf. Table 6).

Table 6:

Scores of CTS group by participants’ L1.

L1ItalianGermanOther L1

Participants who did not indicate German or Italian as their first language achieved the highest score. It is not known if they were foreign students living in Austria or Austrians with a different L1. In both cases, it can be assumed that these participants were more used to creatively using their multilingual repertoires and might thus Lumion 10.0.1 Crack Archives been better at understanding such strategies employed by the speaker.

3.6.5 English proficiency

The influence of listeners’ English proficiency on the intelligibility of ELF users was tested using various results. First, it was evaluated based on the participants’ self-assessment of their English skills. In the questionnaire participants were asked to rate their English listening skills on a 7-point scale from 1 = very good to 7 = very poor. The results indicate good, above-average listening skills (cf. Table 7).

Table 7:

Self-reported English listening skills.


The self-reported English skills of CTS and FH2 participants showed only a low to very low correlation with their test scores. For FH1, however, results of a Spearman correlation indicated that there was a significant moderate correlation between self-assessment of language skills and the test score (rs = −0.47, n = 27, p = 0.007). Thus, in one of the groups, participants with a better knowledge of English were able to achieve better results.

Furthermore, the influence of English proficiency on the score was assessed using additional data. Some of the respondents from FH1 attended a bilingual version of their MA programme with a strict selection procedure at the beginning of the course and many classes in English. One can assume that they were more proficient in English than the other FH1 students. A comparison of their mean values in the listening comprehension test shows that the score of the bilingual students was more than half a point above that of the German-speaking students (cf. Table 8). However, this difference was not statistically significant.

Table 8:

Score FH1 by course type.


These results indicate that in this study, the English proficiency of the participants had only a limited effect on the comprehension test scores and their understanding of the ELF user.

4 Conclusion

The goal of this paper was to gain knowledge about ELF intelligibility that would be relevant for interpreters, in particular for the interpreting sub-skill of comprehension in ELF settings. Since interpreters often identify ELF users as a challenge, the article first reviewed findings regarding the intelligibility of ELF users. The main focus was on listener variables affecting ELF intelligibility, i.e. variables that interpreters, in their capacity as intermediary listeners, themselves can influence in order to improve their performance in ELF settings. Furthermore, the paper presented results from a study that explored intelligibility factors in a conference interpreting setting, i.e. a specialised event with monologic communication that did not offer the possibility to apply common ELF communication strategies such as the negotiation of meaning or clarification procedures.

Since listening and comprehension are skills that are not unique to interpreting, the paper considered findings from all listening contexts that resemble interpreting settings to be relevant for the analysis.

In the author’s study with 67 participants, frequent interactions using ELF and thus experience in ELF settings were found to be a listener variable that strongly affected intelligibility. This finding is in line with studies in various fields of Redshift Render Mac Torrent Archives, where several research reports have shown that listeners’ familiarity with ELF use increases comprehension (see Section 2.1.3). Nevertheless, in this study, familiarity with the speaker’s specific accent did not positively influence intelligibility.

Surprisingly, the variable of background knowledge or expertise was not found to have a strong impact on intelligibility. In the comprehension test, non-expert (interpreting) students achieved higher scores than the students from the speaker’s field of expertise, even though the former stated to be less familiar with the subject. This suggests that the mere fact of belonging to a certain group of experts or having a general knowledge of a topic does not automatically guarantee better understanding of innovative sub-topics such as the one presented by the Italian speaker in the study and that only very specific prior knowledge leads to intelligibility benefits. Thus, in monolingual ELF communication situations, groupness seems to require more than the mere affiliation to a discipline.

Furthermore, research has shown that professional interpreters possess highly efficient comprehension strategies that allow them to carry out a deeper semantic analysis than untrained bilinguals or native speakers. This might imply a more automatic way of performing linguistic analyses in interpreters (Yudes et al. 2013). It is possible that the non-expert interpreting students had already developed better listening comprehension skills than the business students and were thus able to outperform the expert listeners.

This might also imply that interpreters with a high level of specific preparation for a technical conference could be able to better understand ELF users than the general (expert) audience and thus provide an added value with their service.

Another explanation for the non-experts’ good performance might be that many of them did not have German as their L1, which means that they were either foreign students living abroad or Austrians with a different L1 than German. This might have increased their awareness of strategies employed by speakers with multilingual repertoires and might have helped them offset the expert students’ subject knowledge edge.

Other listener variables, such as participants’ English proficiency, only had a moderate influence on comprehension.

As for the variable of a first language shared by speakers and listeners that is often mentioned as a factor facilitating comprehension, only five participants had the same L1 as the speaker and thus no valid conclusions can be drawn in that respect.

It is evident that further research into ELF intelligibility in interpreting settings is needed. The findings of the study do, however, suggest that training of the interpreting part-task of reception in ELF contexts should focus on a maximum of exposure of (future) interpreters to ELF settings and users. In training programmes, a specific emphasis might be put on comprehension tasks and interpreting exercises with non-standard accents and similects that are likely Inovative Logic Crosstrainer II v4.5.7 English by EViDENCE be found in students’ future career settings with regard to regional specificities, markets and employers such as international organisations.

The findings suggest that in mediated ELF contexts, very specific and in-depth knowledge of the topic in question might be necessary. Thus, the students’ research and preparation skills should be geared towards an extremely targeted way of dealing with preparatory documents.

There is every reason to believe that interpreters will encounter an ever-increasing number of ELF users as speakers in their work. Thus, interpreting studies must seek to develop an evidence base for didactic approaches to interpreting in ELF contexts. It is to be hoped that the results of this study will bring us one step closer to this objective.

Corresponding author: Karin Reithofer, Centre for Translation Studies, University of Vienna, Gymnasiumstraße 50, 1180 Vienna, Austria, E-mail:


Albl-Mikasa, Michaela. 2013. Express-ability in ELF communication. Journal of English as a Lingua Franca 2(1). 101–122. in Google Scholar

Albl-Mikasa, Michaela. 2018. ELF and translation/interpreting. In Jennifer Jenkins, Will Baker & Martin Dewey (eds.), The Routledge handbook of English as a Lingua Franca, 369–383. Abingdon, NY: Routledge.Search in Google Scholar

Albl-Mikasa, Michaela, Livia Bartels, Lorenz Mohler & Bernd Wick. 2017. World Englishes and English as a lingua franca in interpreter training. In Susanne Hagemann, Thomas Kempa, Julia Neu & Stephan Walter (eds.), Translationslehre und Bologna-Prozess: Unterwegs zwischen Einheit und Vielfalt, 217–240. Berlin: Frank und Timme.Search in Google Scholar

Anderson-Hsieh, Janet, Ruth Johnson & Kenneth Koehler. 1992. The relationship between native speaker judgments of non-native pronunciation and deviance in segmentals, prosody and syllable structure. Language Learning 42(4). 529–555. in Google Scholar

Bent, Tessa & Ann R. Bradlow. 2003. The interlanguage speech intelligibility benefit. Journal of the Acoustical Society of America 114(3). 1600–1610. in Google Scholar

Berne, Jane. 1993. The role of text type, assessment task and target language experience in L2 listening comprehension assessment. In Paper presented at the Annual Meeting of the American Association Inovative Logic Crosstrainer II v4.5.7 English by EViDENCE Applied Linguistics and the American Association of Teachers of Spanish and Portuguese 1992. (accessed 21 September 2020).Search in Google Scholar

Björkman, Beyza. 2011. Pragmatic strategies in English as an academic lingua franca: Ways of achieving communicative effectiveness? Journal of Pragmatics 43. 950–964. in Google Scholar

Bulatović, Stefan, Anja Schüppert & Charlotte Gooskens. 2019. Receptive multilingualism versus ELF: How well do Slovenes understand Croatian compared to Croatian speakers’ English? Journal of English as a Lingua Franca 8(1). 37–65. in Google Scholar

Chang, Chia-Chien & Michelle, Min-Chia Wu. 2014. Non-native English at international conferences: Perspectives from Chinese-English conference interpreters in Taiwan. Interpreting 16(2). 169–190. in Google Scholar

Cheung, Andrew K. F. 2015. Accent. In Franz Pöchhacker (ed.), Routledge encyclopedia of interpreting studies, 1–2, Inovative Logic Crosstrainer II v4.5.7 English by EViDENCE. London & New York: Routledge.Search in Google Scholar

Clarke, Constance M. 2000. Perceptual adjustment to foreign-accented English. Journal Inovative Logic Crosstrainer II v4.5.7 English by EViDENCE the Acoustical Society of America 107(5). 2856. in Google Scholar

Clarke, Constance M. 2004. Rapid adaptation to foreign-accented English. Journal of the Acoustical Society of America 116(6). 3647–3658. in Google Scholar

Derwing, Tracey & Murray Munro. 1997. Accent, intelligibility and comprehensibility. Evidence from four L1s. Studies in Second Language Acquisition 20(1). 1–16. in Google Scholar

Deterding, David. 2013. Misunderstandings in English as a lingua franca. Berlin: Mouton de Gruyter.Search in Google Scholar

Díaz Galaz, Inovative Logic Crosstrainer II v4.5.7 English by EViDENCE, Stephanie, Presentación Padilla & María Terea, Bajo. 2015. The role of advance preparation in simultaneous interpreting: A comparison of professional interpreters and interpreting students. Interpreting 17(1). 1–25. in Google Scholar

Gass, Susan & Evangeline Varonis. 1984. The effect of familiarity on the comprehensibility of nonnative speech. Language Learning 34(1). 65–89. in Google Scholar

Guido, Maria Grazia. 2012. ELF authentication and accommodation strategies in crosscultural immigration encounters. Journal of English as a Lingua Franca 1(2). 219–240, Inovative Logic Crosstrainer II v4.5.7 English by EViDENCE. in Google Scholar

Hansen Edwards, Jette G., Mary L. Zampini & Caitlin Cunningham. 2018. The accentedness, comprehensibility and intelligibility of Asian Englishes. World Englishes 37. 538–557. in Google Scholar

Herbert, Jean. 1952. The interpreter’s handbook. How to become a conference interpreter. Geneva: Librairie de l’Université Georg.Search in Google Scholar

Huh, Jiun. 2017. Phonological considerations of World Englishes in interpreter training: Pedagogical suggestions based on an experimental study of consecutive interpretation. The Interpreter and Translator Trainer 11(1). 56–78. in Google Scholar

Jenkins, Jennifer. 2000. The phonology of English as an international language: New models, new norms, new goals. Oxford: Oxford University Press.Search in Google Scholar

Kachru, Yamuna. 2008. Cultures, contexts, and interpretability. World Englishes 27(3/4). 309–318. in Google Scholar

Kalina, Sylvia. 2005. Quality assurance for interpreting processes. Meta 50(2). 769–784. in Google Scholar

Kankaanranta, Anne & Leena Louhiala-Salminen. 2018. ELF in the domain of business-BELF. What does the B stand for? In Jennifer Jenkins, Will Baker & Martin, Dewey (eds.), The Routledge handbook of English as a lingua franca, 309–320. London & New York: Routledge.Search in Google Scholar

Katikos, Rena. 2015. Simultaneous interpreting and English as a lingua franca in view of the interlanguage speech intelligibility benefit. Vienna: University of Vienna MA thesis. (accessed 23 September 2019).Search in Google Scholar

Kennedy, Sara. 2017, Inovative Logic Crosstrainer II v4.5.7 English by EViDENCE. Using stimulated recall to explore the use of communication strategies in English lingua franca interactions. Journal of English as a Lingua Franca 6(1). 1–27. in Google Scholar

Kennedy, Sara & Pavel Trofimovich. 2008. Intelligibility, comprehensibility, and accentedness of L2 speech: The role of listener experience and semantic context. Canadian Modern Language Review/La Revue canadienne des langues vivantes 64(3). 459–489. in Google Scholar

Kraus, Julia Maria. 2011. ELF als dolmetschwissenschaftliches Problem. Versuch einer Sensibilisierung auf L1-Aussprachemerkmale spanischsprachiger Redner. Leipzig: Universität Leipzig MA thesis.Search in Google Scholar

Kurz, Ingrid & Elvira Basel. 2009. The impact of non-native English on information transfer in simultaneous interpretation. Forum 7(2). 187–213. in Google Scholar

Lee, Jeong-In, Inovative Logic Crosstrainer II v4.5.7 English by EViDENCE. 2004. A study of L1 influence on speech intelligibility between NNS from various L1. Guildford: University of Surrey MA thesis.Search in Google Scholar

Lin, I-hsin Iris, Fend-lan Ann Chang & Feng-lan Kuo. 2013, Inovative Logic Crosstrainer II v4.5.7 English by EViDENCE. The impact of non-native English on rendition accuracy in simultaneous interpreting. Translation & Interpreting 5(2). 30–44. in Google Scholar

Matsuura, Hiroko, Sarah Rilling, Reiko Chiba, Eun-Young Julia Kim & Nur Rini. 2017. Intelligibility and comprehensibility in English as a lingua franca: Nativized English in Japanese. Asian Englishes 19(1). 2–21. in Google Scholar

Mauranen, Anna. 2018. Conceptualising ELF. In Jannifer Jenkins, Will Baker & Martin Dewey (eds.), The Routledge handbook of English as a lingua franca, 7–24. Abingdon, NY: Routledge.Search in Google Scholar

McNamara, Danielle S., Tenaha O’Reilly & Manuel, de Vega. 2007. Comprehension skill, inference making, and the role of knowledge. In Franz Schmalhofer & Charles A. Perfetti (eds.), Higher level language processes in the brain: Inference and comprehension processes, 233–253. Mahwah, NJ: Erlbaum.Search in Google Scholar

Munro, Murray & Tracey Derwing, Inovative Logic Crosstrainer II v4.5.7 English by EViDENCE. 1999. Foreign accent, comprehensibility and intelligibility in the speech of second language learners. Language Learning 45(1). 73–97.Search in Google Scholar

Munro, Murray, Tracey Derwing & Susan Morton. 2006. The mutual intelligibility of L2 speech. Studies in Second Inovative Logic Crosstrainer II v4.5.7 English by EViDENCE Acquisition 28. 111–131. in Google Scholar

Padilla, Presentación & María HitFilm Pro Activate Full version Archives Bajo. 2015. Comprehension. In Franz Pöchhacker (ed.), Routledge encyclopedia of interpreting studies, 70–73. London & New York: Routledge.Search in Google Scholar

Pickering, Lucy. 2006. Current research in intelligibility in English as a lingua franca. Annual Review of Applied Linguistics 26. 219–233. in Google Scholar

Poncini, Gina. 2004. Discursive strategies in multicultural business meetings. Frankfurt am Main, et al.: Peter Lang.Search in Google Scholar

Reithofer, Karin, Inovative Logic Crosstrainer II v4.5.7 English by EViDENCE. 2010. English as a lingua franca vs. interpreting – Battleground or peaceful co-existence. The Interpreters’ Newsletter 15. 143–157. in Google Scholar

Reithofer, Karin. 2013. Comparing modes of communication – The effect of English as a lingua franca vs. interpreting. Interpreting 15(1). 48–73. in Google Scholar

Reithofer, Karin. 2014. Englisch als Lingua Franca und Dolmetschen. Ein Vergleich zweier Kommunikationsmodi unter dem Aspekt der Wirkungsäquivalenz (Reihe Translationswissenschaft). Tübingen: Gunter Narr Verlag.Search in Google Scholar

Reithofer, Karin. 2018. Interpreting and multilingualism in the EU: Leave or remain? Clina 4(1). 115–136. (accessed 19 September 2019).Search in Google Scholar

Setton, Robin & Andrew Dawrant. 2016. Conference interpreting: A trainer’s guide. Amsterdam & Philadelphia: John Benjamins.Search in Google Scholar

Smith, Larry E. 1992. Spread of English and issues of intelligibility. In Braj B. Kachru (ed.), Inovative Logic Crosstrainer II v4.5.7 English by EViDENCE, The other tongue: English across cultures, 75–90. Urbana: University of Illinois Press.Search in Google Scholar

Smith, Larry E. & Khalilullah Rafiqzad. 1979. English for cross-cultural communication: The question of intelligibility. Tesol Quarterly 13(3). 371–380. in Google Scholar

Tauroza, Steve & Ching Man Luk. 1996. Accent, listening comprehension and low proficiency ESL learners in Hong Kong. Perspectives: Working papers of the Department of English: City Polytechnic of Hong Kong 8(2). 111–134.Search in Google Scholar

Thir, Veronika. Forthcoming. The role of co-text and contextual cues for intelligibility in ELF interactions. In Anna Mauranen & Svetlana Vetchinnikova (eds.), Language change: The impact of English as a lingua franca. Cambridge: Cambridge University Press.Search in Google Scholar

Wang, Hongyan & Vincent J. Van Heuven. 2014. Is a shared interlanguage beneficial? Mutual intelligibility of American, Dutch and Mandarin speakers of English. In Rias van den Doel & Laura Rupp (eds.), Pronunciation matter. Accents of English in the Netherlands and elsewhere, 175–194. Amsterdam: VU University Press.Search in Google Scholar

Wang, Hongyan & Vincent J. Van Heuven. 2015. Inovative Logic Crosstrainer II v4.5.7 English by EViDENCE interlanguage speech intelligibility benefit as bias toward native-language phonology. i-Perception 6(6). 1–13. in Google Scholar

Weil, Shawn A. 2001. Foreign accented speech: Encoding and generalization. Journal of the Acoustical Society of America 109(5). 2473. in Google Scholar

Wilang, Jeffrey Dawala & Adisa Teo. 2012a. Measuring the comprehensibility of Englishes within ASEAN among Aseans. International Journal of Office Recovery Professional 2.0 crack serial keygen and Literature (IJEL) 2(3). 22–42. in Google Scholar

Wilang, Jeffrey Dawala & Adisa Teo. 2012b. Enhancing comprehensibility among ELF users. Journal of English and Literature 2(2). 43–58.Search in Google Scholar

Willis, Gordon B. 2005. Cognitive interviewing: A tool for improving questionnaire design. Thousand Oaks, London, New Delhi: Sage Publications.Search in Google Scholar

Wingsted, Maria & Richard Schulman. 1987. Comprehension of foreign accents. In Wolfgang Dressler AIM Tools Pro Network v3.0 crack serial keygen, Phonologica 1984. Proceedings of the fifth international phonology meeting, 339–345. Cambridge: Cambridge University Press.Search in Google Scholar

Yudes, Carolina, Pedro Macizo, Luis Morales & María Teresa Bajo. 2013. Comprehension and error monitoring in simultaneous interpreters. Applied Linguistics 34(5). 1039–1057. in Google Scholar

Yule, George, Susan Wetzel & Laura Kennedy. 1990. Listening perception accuracy of ESL learners as a variable function of speaker L1. Tesol Quarterly 24(3). 519–523. in Google Scholar

Published Online: 2020-12-03

Published in Print: 2020-09-25

© 2020 Karin Reithofer, published by De Gruyter, Berlin/Boston

This work is licensed under the Creative Commons Attribution 4.0 International License.

Источник: []

Proceedings of the Fifth Biennial Conference of the Society for Implementation Research Collaboration (SIRC) 2019: where the rubber meets the road: the intersection of research, policy, and practice - part 2

  • Meeting abstracts
  • Open Access
  • Published:

Seattle, WA, USA. 12-14 September 2019

Implementation Sciencevolume 15, Inovative Logic Crosstrainer II v4.5.7 English by EViDENCE, Article number: 80 (2020) Cite this article

  • 3201 Accesses

  • 2 Altmetric

  • Metrics details

About this supplement

This article has been published as part of Implementation Science Volume 15 Supplement 2, 2020: Proceedings of the Fifth Biennial Conference of the Society for Implementation Research Collaboration (SIRC) 2019: Where the rubber meets the road: The intersection of research, policy, and practice - Part 2. The first part of this supplement is available online at Please note that this is part 2 of 2.

A1 Public opinion as an outer-contextual factor in health policy D&I research and practice: evidence that the public cares about evidence

Correspondence: Jonathan Purtle (

Dornsife School of Public Health, Inovative Logic Crosstrainer II v4.5.7 English by EViDENCE, Drexel University, Philadelphia, PA, USA


Barriers to evidence-informed health policymaking are well-established [1]. Although many barriers are technical in nature (e.g., poor communication of research findings) [2], a major impediment stems from the political nature of policymaking [3-4]. Public opinion is a key aspect of politics; and one that is relevant to efforts to promote evidence-informed policymaking because public opinion influences policymakers’ behaviors [5-6]. Thus, if policymakers learn that the public wants their decisions to be supported by evidence, this information could spur policymakers to make more evidence-informed health policy decisions and demonstrate evidence use to their constituents. However, no prior research Inovative Logic Crosstrainer II v4.5.7 English by EViDENCE examined public opinion about evidence-informed policymaking. This study sought to characterize public opinion about the influence that evidence should, and does, have on health policy development in U.S. Congress relative to other factors and examine differences in opinion by political party affiliation.

Materials and Methods

A public opinion survey was conducted in 2018 using the SSRS Probability Panel (N=532), a nationally representative internet panel. Respondents separately rated the extent to which six factors (e.g., evidence, budget impact, industry interests) “should have” and “currently have” influence on U.S. congresspersons’ health policy decisions.


Evidence (59%) was the most frequently identified factor that should have “a lot of influence” on health policy development, but only 11% of respondents thought that evidence currently has “a lot of influence” (p<.001). Opinions about evidence did not vary significantly by respondent political party affiliation.


There is strong bi-partisan public support for evidence to have much more influence on health policymaking in U.S. Congress. This finding is promising in a time of elevated political polarization in the United States. The survey results have implications for interventions that aim to promote evidence-informed health policymaking, Inovative Logic Crosstrainer II v4.5.7 English by EViDENCE. As findings suggest public demand for evidence-informed health policymaking, and prior research demonstrating that public opinion often influences elected policymakers’ behaviors [5-6], interventions that systematically document the extent to which elected policymakers’ actions (e.g., public statements, content of bills introduced, tweets) are evidence-supported and disseminate this information to the public could encourage policymakers to make more evidence-informed health policy decisions.


1. National Research Council. Using science as evidence in public policy. Schwandt TA, Straf ML, editors. Washington, DC: The National Academies Press; 2012. doi:

2. Oliver K, Innvar S, Lorenc T, Woodman J, Thomas J. A systematic review of barriers to and facilitators of the use of evidence by policymakers. BMC Health Serv Res. 2014;14(1):2.

3. Liverani M, Hawkins B, Parkhurst JO. Political and institutional influences on the use of evidence in public health policy: a systematic review. PloS One. 2013;8(10):e77404.

4, Inovative Logic Crosstrainer II v4.5.7 English by EViDENCE. Oliver TR. The politics of public health policy. Annu Rev Public Health. 2006;27:195-233.

5. Burstein P. The impact of public opinion on public policy: a review and an agenda. Polit Research Q. 2003;56(1):29-40.

6. Butler DM, Nickerson DW. Can learning constituency opinion affect how legislators vote? Results from a field experiment. Q J Polit Sci. 2011;6(1):55-83.

A2 Federal mental health legislation: what becomes law and why? Results from a 30-year review

Correspondence: Max Crowley (

College of Health and Human Development, Penn State University, University Park, PA, USA


Mental health problems affect millions of individuals and cost over $240 billion annually in increased healthcare, criminal justice, child welfare, education, and labor costs [1]. Ongoing efforts to take evidence-based mental health strategies to scale have encountered a number of barriers to successful uptake and sustained use [2-4]. To overcome these barriers, researchers and advocates often seek to influence public policy to facilitate increased investment in mental health [5-7]. Yet, little work has systematically sought to understand the specific content of past mental health legislation and how the content of the legislation is related to whether a bill becomes law.

Materials and Methods

In order to answer these pressing questions about mental health policy, we conducted a mixed methods review of all federal bills introduced to Congress over the last three decades (1989-2019; N=171,861). This includes systematic coding of mental health’s inclusion in federal legislation, quantitative analyses of that inclusion’s relationship with bills becoming law, and qualitative analyses of how mental health policy may be used to improve population health.


In the 101st Congress (January 3, 1989, to January 3, 1991), only 14 mental health bills were introduced, comprising only .001% of all bills introduced, Inovative Logic Crosstrainer II v4.5.7 English by EViDENCE. By the 115th Congress (January 3, 2017, to January 3, 2019), over 4% of all bills of any type included mental health provisions. In addition to increases in Inovative Logic Crosstrainer II v4.5.7 English by EViDENCE health policy across time, we will also present results of analyses that identify characteristics of bills that are more likely to be successfully enacted into law. Finally, results from qualitative analyses will be used to illustrate how elements of mental health policies can facilitate or restrict high-quality implementation.


We will discuss implications of these findings through the lens of the outer context of the implementation ecology; specifically, by identifying supports and barriers from federal legislation and policy that may be most likely to promote successful implementation of evidence-based treatments. We will also discuss implications for how best to engage directly with policymakers to support increased availability and effectiveness of mental health services.


1. O’Connell ME, Boat TF, Warner KE. Preventing mental, emotional, and behavioral disorders among young people: progress and possibilities. Washington, D.C.: National Academies Press; 2009.

2. CDC. The Power of Prevention. Atlanta, GA: Centers for Disease Control and Prevention; 2009.

3. Haskins R, Margolis G. Show me the evidence: Obama’s fight for rigor and evidence in social policy. Washington, DC: Brookings Institution Press; 2015.

4. Baron J, Haskins R. The Obama administration’s evidence-based social policy initiatives: An overview. 2011.

5. Lynch JP, Sabol WJ. Assessing the effects of mass incarceration on informal social control in communities. Criminol Public Pol. 2004;3(2):267-294. doi:10.1111/j.1745-9133.2004.tb00042.x

6. Ratzliff A, Phillips KE, Sugarman JR, Unützer J, Wagner EH. Practical approaches for achieving integrated behavioral health care in primary care settings. Am J Med Qual. 2017;32(2):117-121. doi:10.1177/1062860615618783

7. Crowley M, Scott T, Fishbein DH. Translating prevention research for evidence-based policymaking: Results from the Research-to-Policy Collaboration pilot. Prev Sci. 2018;19(2):260-270.

A3 Measurement infrastructure for influencing the outer context: Integrating evidence-based practice reporting and client surveys to guide decision-making in a learning health care system

Noah R. Gubner1,2, Felix I. Rodriguez3, Rose Krebill-Prather4, Kristen Petersen4, Sarah Cusworth Walker1,2

1Department of Psychiatry and Behavioral Sciences, University of Washington, Seattle, WA, USA; 2Washington State Evidence-Based Practice Institute, Seattle, WA, USA; 3Washington State Health Care Authority, Olympia, WA, USA; 4Social and Economic Sciences Research Center, Washington State University, Pullman, WA, USA

Correspondence: Sarah Cusworth Walker (


Legislation in Washington State (HB2536), passed in 2012, mandated the reporting of evidence-based practices (EBPs) for all children-serving systems in Washington State, Inovative Logic Crosstrainer II v4.5.7 English by EViDENCE. In response, the Division of Behavioral Health and Recovery and University of Washington’s Evidence-Based Practice Institute developed a measurement method to track use of EBPs in routine services statewide. This method provides a cost effective and adaptable surveillance tool to monitor evidence-based practices (EBPs) and can also be merged with other data sources to monitor disparities in utilization and effectiveness. In this paper we will present this as an example of “outer setting” influences on EBP use as well as proof of concept for this integrated data infrastructure as a means to guide decision-making at the policy level.

Materials and Methods

We analyzed a sample of youth (<21 years old) from Washington State who had received at least one hour of publicly funded outpatient mental health (MH) service. Current procedural terminology (CPT) billing codes were used to identify if clients in the sample received any valid EBP psychotherapy sessions during the observation period (May – Oct 2015). Billing data were then tied to self-reported perceptions of outcomes of service from the Inovative Logic Crosstrainer II v4.5.7 English by EViDENCE Mental Health Consumer Survey. These outcome data were compared between youth who did and did not receive a valid EBP session during the observation period


Among this sample of 1,580 youth, 19.7% (n=312) received at least one valid EBP psychotherapy session. Youth from rural (21.4%) versus urban (16.2%) providers were more likely to have received an EBP session (χ2=6.02, p=0.014). There were significant differences by race/ethnicity (χ2=14.71, p=0.04). Non-Hispanic Whites (20.2%) and American Indians (33.3%) were more likely, while African Americans (12.3%) and Hispanics (15.7%) were less likely to have received a valid EBP session. There was a trend for an interaction between race/ethnicity and receipt of an EBP session on self-reported positive services outcomes, with receipt of an EBP session potentially being associated with more positive services outcomes among youth who were non-White versus White.


As a proof of concept, we demonstrate that billing data provides a cost-effective tool to monitor the receipt of EBP MH sessions and can be tied to other data sources to examine outcomes across a health network.

A4 Giving the outer setting its due: adapting the stages of implementation completion to policy and system-level change efforts

Eric J. Bruns1, Jonathan R. Olson1, Philip H. Benjamin1, Lisa Saldana2

1Department of Psychiatry & Behavioral Sciences, University of Washington, Seattle, WA, USA; 2Oregon Social Learning Center, Eugene, OR, USA

Correspondence: Eric J. Bruns (


Successful implementation of Wraparound care coordination for youth with complex behavioral health needs requires hospitable policy and financing conditions [1]. Thus, when the “rubber meets the road,” implementation support for Wraparound requires attention to the “outer setting” of the implementation ecology [2], including cross-agency coordination, Medicaid payment reform, and cross-sector information systems. Unfortunately, Inovative Logic Crosstrainer II v4.5.7 English by EViDENCE, existing implementation measures are scarce at the outer context [3], as is research on relevant outer setting strategies [4]. This presentation describes efforts to track Wraparound implementation at system and organizational levels using an adaptation of the Stages of Implementation Completion (SIC) [5].

Materials and Methods

The SIC assesses implementation across eight stages and three phases: pre-implementation, implementation, and sustainment. We adapted Inovative Logic Crosstrainer II v4.5.7 English by EViDENCE added SIC items to align with multilevel Wraparound implementation support as enacted by the National Wraparound Implementation Center (NWIC), including an array of state systems (outer setting) variables such as state leadership engagement, cross-system communication and collaboration, financing strategies, and contract requirements. We have collected data for two pilot states and are completing collation for eight additional states.


Adaptation of the SIC entailed adding 12 items and removing 4. Furthermore, we operationalized each SIC variable in terms of measurable Wraparound processes and activities. Preliminary results from two pilot states indicate high completion rates across stages for both states (completion percentages from 60% to 100%). However, Inovative Logic Crosstrainer II v4.5.7 English by EViDENCE, states differed significantly in their time to completion, with State 1 averaging 3.75 months for completion of stages and state 2 averaging 26.38 months. Item- and stage-level analyses revealed that State 2 struggled to engage state leadership in implementation. State 1 adopted a new approach to building Wraparound implementation infrastructure – investing in Care Management Entities (CMEs) [6], while State 2 relied on Community Mental Health Centers.


Findings provide proof of concept for incorporating outer setting items into an established implementation measure and underscore the influence of outer context in Wraparound implementation. The presentation will show how NWIC is using this measure to support states to build systems that facilitate better implementation and outcomes, while also promoting new and needed research for mental health and implementation science.


1. Bruns EJ, Sather A, Pullmann MD, Stambaugh LF. National trends in implementing wraparound: results from the state wraparound survey. J Child Fam Stud. 2011;20(6):726-735.

2. Damschroder LJ, Aron DC, Keith RE, Kirsh SR, Alexander JA, Lowery JC. Fostering implementation of health services research findings into practice: a consolidated framework for advancing implementation science. Implement Sci. 2009; 4:50. doi:10.1186/1748-5908-4-50.

3. Lewis CC, Stanick CF, Martinez RG, Weiner BJ, Kim M, Barwick M, Comtois KA. The Society for Implementation Research Collaboration Instrument Review Project: a methodology to promote rigorous evaluation. Implement Sci. 2015; 10:2.

4. Purtle J, Peters R, Brownson RC. A review of policy dissemination and implementation research funded by the National Institutes of Health, 2007–2014. Implement Sci. 2016;11:1.

5. Saldana L, Chamberlain P, Wang W, Brown CH. Predicting program start-up using the stages of implementation measure. Adm Policy Ment Health. 2012:39(6):419-25.

6. Center for Health Care Strategies. Care management entities: a primer. 2011. Accessed 27 March 2019.

A5 Cross-collaborations among researchers, community, government agencies, and a federal funding agency to support implementation of evidence-based cardiovascular disease prevention in primary care: the EvidenceNOW Initiative

Donna Shelley1, Michael Parchman2, Robert McNellis3

1School of Medicine, New York University, New York, NY, USA; 2Kaiser Permanente Washington Health Research Institute, Seattle, WA, USA; 3Agency for Healthcare Research and Quality, Rockville, MD, USA

Correspondence: Donna Shelley (


EvidenceNOW is an Agency for Healthcare Research and Quality (AHRQ)-funded research initiative focused on helping primary care practices improve the delivery of the “ABCS” of cardiovascular disease (CVD) prevention: Aspirin in high-risk individuals, Blood pressure control, Cholesterol management, and Smoking cessation. Seven cooperatives used practice facilitation (PF) as a unifying strategy to support the implementation and dissemination of evidence-based care for CVD risk factors. Each cooperative enrolled over 200 practices in their region. Cooperatives created an infrastructure to engage stakeholder organizations in their region to facilitate the initiative. AHRQ created infrastructure that included a national evaluator (ESCALATES) and committees and meetings that fostered cross-cooperative collaboration and dissemination.

Materials and Methods

This panel includes an AHRQ program officer who will discuss the infrastructure created and lessons learned for guiding the design of future funding opportunities [1] and case studies demonstrating how two cooperatives, HealthyHearts Coolutils Total Doc Converter [] Crack Download Latest Version York City [2] and Healthy Hearts Northwest [3], partnered with community and government agencies in their region to accomplish EvidenceNOW goals. The discussant is the ESCALATES Principal Investigator.


The aligned efforts of Cooperatives, AHRQ, and ESCALATES facilitated the collective expansion of capacity for practice transformation and rigorous evaluation. We found that: (a) large-scale, federally-funded research initiatives were strengthened through infrastructure that fostered collaboration across grantees (demonstrated by 12 cross-cooperative publications to date); (b) an external evaluator added tremendous value in supporting cross-cooperative collaboration and amplified opportunities for dissemination; (c) engaging with partner organizations early helped assess fit with organizational strategic plans, capacity, readiness for change, and data collection systems; (d) applying an implementation science framework [4] was necessary to guide intervention development and assessment; and (e) stakeholder organizations valued being included in research and funder meetings and dissemination Inovative Logic Crosstrainer II v4.5.7 English by EViDENCE cooperatives, aligning goals, and attending to opportunities to grow research and quality improvement capacity among partnering agencies facilitated strong, enduring partnerships for practice transformation. Funders have a role to play in facilitating collaborations among cooperatives and evaluation. The findings that emerge through the efforts of multi-level partners are greater than the sum of the parts and further the field of dissemination and implementation science.


1. Cohen DJ, Balasubramanian BA, Gordon L, Marino M, Ono S, Solberg LI, Crabtree BF, Strange KC, Davis M, Miller WL, Damschroder LJ, McConnell KJ, Creswell J. A national evaluation of a dissemination and implementation initiative to enhance primary care practice capacity and improve cardiovascular disease care: the ESCALATES study protocol. Implement Sci. 2016;11(86):1-13. doi:10.1186/s13012-016-0449-8

2. Shelley DR, Ogedegbe G, Anane S, Wu WY, Goldfield K, Gold HT, Kaplan S, Berry C. Testing the use of practice facilitation in a cluster randomized stepped-wedge design trial to improve adherence to cardiovascular disease prevention guidelines: HealthyHearts NYC. Implement Sci. 2015;11(1):88. doi:10.1186/s13012-016-0450-2

3. Parchman ML, Fagnan LJ, Dorr DA, Evans P, Cook AJ, Penfold RB, Hsu C, Cheadle A, Baldwin LM, Tuzzio L, Inovative Logic Crosstrainer II v4.5.7 English by EViDENCE. Study protocol for “Healthy Hearts Northwest”: a 2 × 2 randomized factorial trial to build quality improvement capacity in primary care. Implement Sci. 2016;11(138):1-9. doi:10.1186/s13012-016-0502-7

4. Damschroder LJ, Aron DC, Keith RE, Kirsh SR, Alexander JA, Lowery JC. Fostering implementation of health services research findings into practice: a consolidated framework for advancing implementation science. Implement Sci. 2009;4(1):50. doi:10.1186/1748-5908-4-50

A6 A collaboration between practitioners, intermediaries, and researchers to increase access to evidence-based chronic pain care

Jessica Chen1,2, Lisa Glynn2, Timothy Dawson1,2, Hannah Gelman3, Steven Zeliadt1,3

1Department of Psychology, University of Washington, Seattle, WA, USA; 2VA Puget Sound, Seattle, WA, USA; 3Seattle-Denver Center of Innovation, VA HSR&D, Seattle, WA, USA

Correspondence: Jessica Chen (


This presentation describes initial lessons learned from a collaboration between front-line providers, intermediaries, and researchers to implement a telehealth model of chronic pain care to rural VA clinics.

As opioid-related overdoses have increased exponentially in the U.S. over the last two decades, there has been greater emphasis on increasing access to non-opioid and non-pharmacological pain management, particularly to rural and under-served areas and in primary care settings.

Materials and methods

To address these priorities, front-line clinicians and clinical administrators at one Veterans Health Administration (VA) facility initiated a collaboration with VA implementation researchers to implement a novel hub-and-spoke telehealth model for chronic pain management (TelePain), which will deliver patient education, evidence-based psychotherapies, movement therapies (e.g., yoga for lower back pain), and non-opioid pharmacotherapies Inovative Logic Crosstrainer II v4.5.7 English by EViDENCE a central specialty pain “hub” to rural VA primary care clinic “spokes.” The goal of the collaboration between practitioners, intermediaries, and researchers was to obtain grant funding to support external facilitation and evaluation activities.


There were several lessons learned from this collaborative process regarding health care system changes that may better foster collaboration between research, policy, and practice. One was that the unpredictable and cyclical nature of grant funding can interfere with delivering implementation support when it is needed, namely at the time of clinical services rollout. Therefore, when clinical programs are being designed, they should consider building in funding for implementation support and evaluation from the outset. A second lesson learned was that implementation intermediaries, individuals who have experience with both care delivery and Inovative Logic Crosstrainer II v4.5.7 English by EViDENCE sciences, Inovative Logic Crosstrainer II v4.5.7 English by EViDENCE, may be particularly important for bridging the cultural divide between researchers and practitioners.


Embedding intermediaries as fully-funded staff in the TelePain clinical program has helped to align research aims with clinical priorities and may support the long-term sustainability of implementation efforts.

A7 Untangling trauma-related knowledge and practice changes among brokers in a community-based learning collaborative: role of interprofessional collaboration

Funlola Are1, Rochelle Hanson1, Samuel Peer2, Ben Saunders1

1Department of Psychiatry and Behavioral Sciences, Medical University of South Carolina, Charleston, SC, USA; 2Psychology Department, Idaho State University, Pocatello, ID, USA

Correspondence: Funlola Are (


While evidence-based treatments (EBTs) exist to ameliorate trauma-related mental health problems, many children do not receive them [1], Inovative Logic Crosstrainer II v4.5.7 English by EViDENCE. Possible reasons to account for this include limited availability of EBTs and poor collaboration amongst professionals involved in youth service provision [2]. Brokers, often child welfare workers, serve an important intermediary role in improving service access for youth [3], Inovative Logic Crosstrainer II v4.5.7 English by EViDENCE, but they are often trained separately from clinical providers, precluding the opportunity to promote cross-discipline collaboration. Community Based Learning Collaboratives (CBLC) use specific training/ implementation strategies involving multidisciplinary stakeholders to foster collaboration and build community capacity for trauma-focused EBTs [4]. The broker curriculum includes information about trauma impact, trauma-focused EBTs, family engagement strategies, and trauma-focused treatment planning, while also providing opportunities for cross-discipline training.

Materials and Methods

This presentation examines changes in trauma-related knowledge, practices and interprofessional collaboration among n = 33 brokers participating in CBLCs conducted as part of a statewide dissemination initiative. Brokers completed self-report measures examining knowledge of trauma-related topics taught as part of the CBLC curriculum, organizational climate, interprofessional collaboration, and trauma-related practices (e.g., assessment, psychoeducation) pre- and post-CBLC. Bivariate correlations were computed in SPSS, and interaction effects were probed using the PROCESS macro in SPSS.


Analyses revealed that increases in knowledge about the trauma-related curriculum topics were significantly associated with positive changes in broker practices following CBLC participation. However, neither Richardson RazorSQL 9.4.6 Crack Full Version Free Download related to interprofessional collaboration nor organizational climate were significantly related to changes in broker practices. Moderation analyses revealed a significant interaction effect between interprofessional collaboration and knowledge of evidence-based treatment planning pre- and post-CBLC [F (1, 29) = 5.14; p = .03; R2 = .25]. Those participants who reported the greatest pre- to post-CBLC change in interprofessional collaboration also reported a significant increase in skills related to evidence-based treatment planning (t = 2.62, p = .03, 95% CI = .04, 0.84).


Study findings suggest that brokers play an important role in building community capacity for EBT access and that cross-discipline strategies help to foster collaborative relationships among youth service providers. Implications for future research, policy and practice will be addressed.


1, Inovative Logic Crosstrainer II v4.5.7 English by EViDENCE. U.S. Department of Health & Human Services, Administration for Children and Families, Administration on Children, Youth and Families, Children’s Bureau. Child maltreatment 2016. 2018.

2. Fitzgerald MM, Torres MM, Shipman K, Gorrono J, Kerns SEU, Dorsey S. Child welfare caseworkers as Getdataback NTFS 4.33 crack serial keygen of mental health services: a pilot evaluation of Project Focus Colorado. Child Maltreat. 2015; 20(1);37-49.

3. Stiffman AR, Pescosolido B, Inovative Logic Crosstrainer II v4.5.7 English by EViDENCE, Cabassa LJ, Inovative Logic Crosstrainer II v4.5.7 English by EViDENCE. Building a model to understand youth service access: the gateway provider model. Ment Health Serv Res. 2004;6:189. 0000044745.09952.33

4. Hanson RF, Saunders BE, Ralston E, Moreland AD, Peer SO, Fitzgerald MM. Statewide implementation of trauma-specific treatment services using the Community-Based Learning Collaborative Model. Psychol Serv. 2019;16:171-181.

A8 Change in patient outcomes after augmenting a low-level implementation strategy in community practices that are slow to adopt a collaborative chronic care model: a cluster randomized implementation trial

Shawna Smith1, Daniel Almirall1, Katherine Prenovost1, Mark Bauer2, Celeste Liebrecht1, Daniel Eisenberg1, Amy Kilbourne1

1Institute for Healthcare Policy and Innovation University of Michigan, Ann Arbor, MI, USA; 2Department of Psychiatry, Harvard Medical School-VA Boston, Boston, MA, USA

Correspondence: Shawna Smith (


Implementation strategies are essential for promoting uptake of evidence-based practices and for patients to receive optimal care [1]. Yet strategies differ substantially in their intensity and feasibility. Lower-intensity strategies (e.g., training, technical support) are commonly used, but may be insufficient for all clinics. Limited research has examined the comparative effectiveness of augmentations to low-level implementation strategies for non-responding clinics [2-3].

Materials and methods

In this Hybrid Type III implementation-effectiveness study [4], we compare the effectiveness of two augmentation strategies, External Facilitation (EF) vs. External + Internal Facilitation (EF/IF)[5] for improving uptake of an evidence-based collaborative care model (CCM) on 18-month mental health outcomes for patients with depression at community-based clinics initially non-responsive to lower-level implementation support.


Providers initially received support using a low-level implementation strategy, Replicating Effective Programs (REP). After 6 months, non-responsive clinics that had failed to deliver a clinically significant dose of the CCM to >10 patients were randomized to augment REP with either EF or EF/IF. Mixed effects models evaluated the comparative effectiveness of the two augmentations on patient outcomes at 18 months. The primary outcome was patient SF-12 mental health score; secondary outcomes were PHQ-9 depression score and self-reported receipt of the CCM during months 6 through 18.

27 clinics were non-responsive after 6 months of REP. 13 clinics (N=77 patients) were randomized to REP+EF and 14 (N=92) to REP+EF/IF. At 18 months, patients in the REP+EF/IF arm had worse SF-12 (diff=8.38; 95%CI=3.59, 13.18) and PHQ-9 scores (diff=1.82; 95%CI=-0.14, 3.79), and lower odds of CCM receipt (OR=0.67, 95% CI=0.30,1.49) than REP+EF patients.


Patients at initially non-responsive community-based clinics that were randomized to receive the more intensive EF/IF augmentation saw less improvement in mood symptoms at 18 months than those the received the EF augmentation, and were also no more likely to receive the CCM. While EF generally appeared to help clinics, a number of EF/IF clinics experienced barriers in implementing the IF strategy with fidelity, including failing to identify an IF. For large-scale implementation in community-based clinics, augmenting REP with EF for sites that need additional support may be more feasible, and ultimately more effective, than a more intensive EF/IF augmentation.

Trial Registration: NCT02151331


1. Proctor EK, Powell Corel VideoStudio Pro X2 crack serial keygen, McMillen JC. Implementation strategies: recommendations for specifying and reporting. Implement Sci. 2013;8:139. doi:10.1186/1748-5908-8-139

2, Inovative Logic Crosstrainer II v4.5.7 English by EViDENCE. Bonham AC, Solomon MZ. Moving comparative effectiveness research into practice: implementation science and the role of academic medicine. Health Aff. 2010;29.10:1901-1905.

3. Atkins D, Kupersmith J. Implementation research: A Inovative Logic Crosstrainer II v4.5.7 English by EViDENCE component of realizing the benefits of comparative effectiveness research. Am J Med. 2010;123(12):e38-e45.

4. Curran GM, Bauer M, Mittman B, Pyne JM, Stetler C. Effectiveness-implementation hybrid designs: combining elements of clinical effectiveness and implementation research to enhance public health impact. Med Care. 2012; 50(3):217-26.

5. Kilbourne, AM, Almirall, D, Eisenberg, D, Waxmonsky, J, Goodrich, Inovative Logic Crosstrainer II v4.5.7 English by EViDENCE, DE, Fortney, JC, Kyle, J. Protocol: Adaptive Implementation of Effective Programs Trial (ADEPT): Cluster randomized SMART trial comparing a Pokémon go crack serial keygen versus enhanced implementation strategy to improve outcomes of a mood disorders program. Implement Sci. 2010;9(1):132.

A9 A community-based implementation roadmap to inform scalability, sustainability, and spread of evidence-based collaborative care interventions

Amy Rusch, Shawna Smith, Lindsay Decamp, Celeste Liebrecht, Gregory Dalack, Amy Kilbourne

Institute for Healthcare Policy and Innovation University of Michigan, Ann Arbor, Inovative Logic Crosstrainer II v4.5.7 English by EViDENCE, USA

Correspondence: Amy Rusch (


Evidence-based collaborative care interventions (CCIs) can improve health by mitigating the access gap in mental health care treatment. However, CCIs can be difficult to implement and Inovative Logic Crosstrainer II v4.5.7 English by EViDENCE to scale up CCIs are often stymied by a lack of Inovative Logic Crosstrainer II v4.5.7 English by EViDENCE knowledge for identifying and addressing barriers to implementation [1]. Implementation roadmaps provide a playbook outlining critical steps practitioners should follow in scaling up CCIs to new settings that overcome implementation barriers, measure implementation success, and garnering leadership support for longer-term CCI sustainability [2]. We present an Implementation Roadmap [3] that guides community-based CCI implementation based on the experiences of stakeholders successfully implementing a broad spectrum of CCIs through the Michigan Mental Health Integration Partnership (MIP) [4]. The goal of MIP is to support the scale up and spread of CCIs that enhance access to care for Medicaid-eligible consumers with behavioral healthcare needs, while also providing an in-situ implementation laboratory for informing sustained uptake of CCIs across a variety of community-based settings.

Materials and Methods

Semi-structured interviews were carried out with stakeholders from successfully adopted MIP CCI projects to define common barriers, challenges, and implementation strategies deployed by the project teams. Interviews were transcribed and analyzed for common themes that identified a series of critical steps scaffolding the CCI implementation process and accompanying metrics for evaluating implementation progress. 25 interviews of key stakeholders were conducted across 7 successful MIP implementation teams, including 11 providers at implementation sites and 14 researchers/project managers.


Stakeholders commonly identified specific steps that overcame barriers to CCI implementation, including deployment of web-based tools for facilitating implementation, embedding key metrics of 3D Earth Screensaver Version: 2.1 crack serial keygen success, garnering upper level administration buy-in upfront, and specifying a process for tailoring implementation strategy deployment to specific site needs. These findings informed our resulting Implementation Roadmap, which includes eleven critical implementation steps and evaluative metrics for investigators implementing CCIs to consider across pre-implementation, implementation, and sustainability phases.


Maximal CCI public health impact requires improved reach. Our Implementation Roadmap provides a clear and practical guide for early stage community CCI implementation efforts, and ensure practitioners collect key metrics and systematically address barriers in ways that are foundational for larger scale, sustainable implementation efforts.


1. Beidas RS, Stewart RE, Adams DR, Fernandez T, Lustbader S, Powell BJ, Aarons GA, Hoagwood KE, Evans AC, Hurford MO, Rubin R, Hadley T, Mandell DS, Barg FK. A multi-level examination of stakeholder perspectives of implementation of evidence-based practices in a large urban publicly-funded mental health system. Adm Policy Ment Health. 2016;43(6):893-908.

2. Proctor E, Luke Inovative Logic Crosstrainer II v4.5.7 English by EViDENCE, Calhoun A, McMillen C, Brownson R, McCrary S, Padek M. Sustainability of evidence-based healthcare: research agenda, methodological advances, and infrastructure support. Implement Sci. 2015;10(88):1-13.

3. Bolboli AS, Reiche M. A model for sustainable business excellence: implementation and the roadmap. TQM J. 2013;25(4), 331-346.

4. Heller D J, Hoffman C, Bindman AB. Supporting the needs of state health policy makers through university partnerships. J Health Polit Policy Law. 2014;39(3):667-677.

A10 The collaborative chronic care model for mental health conditions: from partnered implementation trial to scale-up and spread

Mark Bauer1,2, Kendra Weaver3, Bo Kim1,2, Christopher Miller1,2, Robert Lew1,4, Kelly Stolzmann1, Jennifer Sullivan1,4, Rachel Riendeau1,5, Samantha Connolly1,2, Jeffery Pitcock6, Stig Ludvigsen1, A. Rani Elwy1,7

1Veterans Health Administration, Boston, MA, USA; 2Department of Psychiatry, Harvard Medical School Boston, MA, USA; 3VA Office of Mental Health and Suicide Prevention, Washington, DC, USA; 4School of Public Health, Boston University; VRay 5 Crack Archives of Anthropology, University of Iowa, Iowa City, IA, USA; 6Central Arkansas Veterans Healthcare System, Little Rock, AR, USA; Inovative Logic Crosstrainer II v4.5.7 English by EViDENCE of Psychiatry and Human Behavior, Brown University Warren Alpert School of Medicine, Providence, RI, USA

Correspondence: Mark Bauer (


Collaborative Chronic Care Models (CCMs) have extensive controlled trial evidence for effectiveness in serious mental illnesses [1-2], but there is little evidence regarding feasibility or impact in typical practice conditions. In partnership with the VA Office of Mental Health and Suicide Prevention (OMHSP) we conducted a randomized, stepped wedge implementation trial using blended internal-external facilitation [3] to implement CCMs in Behavioral Health Interdisciplinary Program (BHIP) teams in the general mental health clinics of nine VA medical centers [4-5]. Based on experience in this trial, OMHSP launched an initiative to scale-up and spread the implementation effort more broadly.

Materials and Methods

Our research team and OMHSP engaged with Transformational Coaches (T-Coaches) from the VA Office of Veterans Access to Care to serve as external facilitators to engage additional VA medical centers across the country. T-Coaches are senior facilitators with skills in team-building and process redesign from diverse professional disciplines, Inovative Logic Crosstrainer II v4.5.7 English by EViDENCE. Trial external facilitators and OMHSP leadership trained 17 T-Coaches in methods used in the trial. Sites were Inovative Logic Crosstrainer II v4.5.7 English by EViDENCE by OMHSP. Blended facilitation was conducted for 12 months as in the implementation trial. Each of the T-Coaches partnered with a BHIP-CCM subject matter expert for the effort, and they conferred on a regular basis throughout the year.


Thirty-nine sites were approached; of these 35 (89.7%) signed a letter of agreement. Of these, 28 facilities (80.0%) completed a site visit and entered the ongoing virtual facilitation process. Of these, 21 facilities (75.0%) completed the one-year facilitation and submitted CCM-concordance process summaries. The proportion of CCM-concordant processes ranged widely across facilities, with the more concordant sites equaling rates seen in the implementation trial and a broader low-end distribution (trial: 44-89, T-Coach scale-up: 13-93%).


In summary there was, not surprisingly, a broader range of CCM-concordance among these scale-up sites compared to the implementation trial. Nonetheless, taken together, the two BHIP-CCM implementation efforts reached 30 VA medical centers, of which 17 (56.7%) aligned over half of designated care processes with the evidence-based CCM. With strong operational partnerships and support, implementation trial efforts can be scaled up and spread to achieve broader healthcare system impact.

Trial Registration: NCT02543840


1. Woltmann E, Grogan-Kaylor A, Perron B, Georges H, Kilbourne AM, Bauer MS. Comparative effectiveness of collaborative chronic care models for mental health conditions across primary, specialty, and behavioral health care settings: systematic review and meta-analysis. Am J Psychiatry. 2012; 169:790-804.

2. Miller CJ, Grogan-Kaylor A, Perron BE, Kilbourne AM, Woltmann E, Bauer MS. Collaborative chronic care models for mental health conditions: cumulative meta-analysis and metaregression to guide future research and implementation. Med Care. 2013;51:922-930.

3. Kirchner JE, Ritchie MJ, Pitcock JA, Parker LE, Curran GM, Fortney JC. Outcomes of a partnered facilitation strategy to implement primary care-mental health. J Gen Intern Med. 2014;29 Suppl 4:904-912.

4. Bauer MS, Miller C, Kim B, Lew R, Weaver K, Coldwell C, Henderson K, Holmes S, Seibert MN, Stolzmann K, Elwy AR, Kirchner J. Partnering with health system operations leadership to develop a controlled implementation trial. Implement Sci. 2016;11:22.

5, Inovative Logic Crosstrainer II v4.5.7 English by EViDENCE. Bauer MS, Miller C, Kim B, Lew R, Stolzman K, Sullivan J, Reindeau R, Pitcock J, Williamson A, Connolly S, Elwy AR, Weaver K. Effectiveness of implementing a Collaborative Chronic Care Model for clinician teams on patient outcomes and health status in mental health: a randomized clinical trial. JAMA Netw Open. 2019;2:e190230.

A11 A randomized stepped wedge hybrid-II trial to implement the collaborative chronic care model in VA general mental health clinics

Christopher Miller1, Bo Kim1, Robert Lew1, Kelly Stolzmann1, Jennifer Sullivan1, Rachel Riendeau2, Jeffery Pitcock3, Alicia Williamson4, Samantha Connolly1, A. Rani Elwy5, Kendra Weaver6, Mark Bauer1

1VA Boston Healthcare System, Boston, MA, USA; 2Department of Anthropology, University of Iowa, Iowa City, IA, USA; 3Central Arkansas Veterans Healthcare System, Little Rock, AR, USA; 4School of Information, University of Michigan, Ann Arbor, MI, USA; 5Brown University Warren Alpert School of Medicine, Providence, RI, USA; 6VA Office of Mental Health and Suicide Prevention, Washington, DC, USA

Correspondence: Christopher Miller (


Collaborative Chronic Care Models (CCMs) have extensive controlled trial evidence for effectiveness in serious mental illnesses [1], Inovative Logic Crosstrainer II v4.5.7 English by EViDENCE, but there is little evidence regarding feasibility or impact in typical practice conditions. We determined the effectiveness of implementation facilitation on establishing the CCM in mental health teams, and its impact on health outcomes of team-treated individuals.

Materials and Methods

We used a randomized stepped wedge trial in Behavioral Health Interdisciplinary Program (BHIP) teams in outpatient general mental health clinics of nine VA facilities, using blended internal-external facilitation. Facilitation combined a study-funded external facilitator with a facility-funded internal facilitator working with a designated team for one year. We hypothesized that facilitation would be associated with improvements in both implementation and intervention outcomes (hybrid-II trial) [2]. Implementation outcomes included the clinician Team Development Measure (TDM) and proportion of CCM-concordant team care processes. The study was powered for the primary health outcome, VR-12 Mental Component Score (MCS). All Veterans treated by designated teams were included for hospitalization analyses, based on administrative data; a randomly selected sample was identified for health status interview. Individuals with dementia were excluded. For implementation outcomes, 62 clinicians were surveyed; site process summaries were rated for CCM concordance.


The population (n=5,596) included 881 (15%) women, average age 52.2+14.5. The interviewed sample (n=1,050) was similar, but oversampled for women (n=210, 20.0%). Facilitation was associated with improvements in TDM subscales for role clarity and team primacy. Percentage of CCM-concordant processes achieved varied (44-89%). No improvement in veteran self-ratings, including the primary outcome, was seen. However, in post-hoc analyses MCS improved in veterans with >3 treated mental health diagnoses versus others. Mental health hospitalization rate demonstrated a robust drop during facilitation; this finding withstood four internal validity tests [3].


Working solely at the clinician level with minimal study-funded support, CCM implementation yielded provider and Veteran benefits. Although impact on self-reported overall population health status was negligible, health status improved for complex individuals, Inovative Logic Crosstrainer II v4.5.7 English by EViDENCE, and hospitalization rate declined. Facilitating CCM implementation provides a potential model for realigning VA outpatient general mental health care with an evidence-based model that improves provider team function and Veteran outcomes.

Trial Registration: NCT02543840


1. Miller CJ, Grogan-Kaylor A, Perron BE, Kilbourne AM, Woltmann E, Bauer MS. Collaborative chronic care models for mental health conditions: cumulative meta-analysis and meta-regression to guide future research and implementation. Med Care. 2013;51:922-930.

2. Bauer MS, Miller CJ, Kim B, Lew R, Weaver K, Coldwell C, Henderson K, Holmes SK, Nealon-Seibert M, Stolzmann K, Elwy AR, Kirchner J. Partnering with health system operations leadership to develop a controlled implementation trial. Implement Sci. 2016;11(22):1-11.

3. Bauer MS, Miller CJ, Kim B, Lew R, Stolzmann K, Sullivan J, Riendeau R, Pitcock J, Williamson A, Connolly S, Elwy AR, Weaver K. Effectiveness of implementing a collaborative chronic care model for clinician teams on patient outcomes and health status in mental health: a randomized clinical trial. JAMA Netw Open. 2019;2(3):e190230. doi:10.1001/jamanetworkopen.2019.0230.

A12 Development, adaptation, and preliminary evaluation of the Leadership and Organizational Change for Implementation strategy

Mark G. Ehrhart1, Marisa Sklar2,3, Kristine Carandang2,3, Melissa R. Hatch2,3, Joanna C. Moullin4, Gregory A. Aarons2,3

1Psychology Department, University of Central Florida, Orlando, FL, USA; 2Department of Psychiatry, University of California San Diego, San Diego, CA, USA; 3Child and Adolescent Services Research Center, University of California San Diego, San Diego, CA, USA; 4Faculty of Health Sciences, Curtin University, Perth, Australia

Correspondence: Gregory A. Aarons (


This presentation describes the development of the Leadership and Organizational Change for Implementation (LOCI) strategy [1], subsequent adaptation, and preliminary data for implementing Motivational Interviewing (MI) in substance abuse treatment settings [2]. LOCI is an implementation strategy developed to align higher level organizational strategies with first-level leader development to create a strategic organizational climate to support implementation and sustainment of evidence-based practices (EBPs) [3]. Adaptations to the general design of LOCI that have occurred over time, as well as adaptations built into LOCI to tailor the strategy to particular settings will be described. LOCI provides a general structure, curricula, and process for leading implementation, allowing for flexibility so that leaders across levels can prioritize issues most relevant at a given time for a given context. We also describe mechanisms of change central to LOCI.

Materials and Methods

The current study involves 3 cohorts of 20 clinics in which clinics are being randomized to LOCI vs, Inovative Logic Crosstrainer II v4.5.7 English by EViDENCE. webinar control conditions. For this presentation we utilize data from the first cohort 19 clinics and quantitatively examine differences in implementation leadership and climate by condition. Mathematica 11.3 activation key Archives data were collected from and analyzed to identify factors influencing EBP implementation. Fourteen leaders across intervention and control conditions responded to an online survey and provided rankings (order of importance) and ratings of factors affecting EBP implementation. These data were supplemented with qualitative interviews.


Preliminary results from the first cohort demonstrated that LOCI, compared to the control condition, showed significant improvements in implementation leadership and implementation climate. Qualitative analyses showed that compared to control leaders, LOCI leaders were more focused on staff competency and overcoming resistance. Time, turnover, and the influence of external contracts emerged as key themes.


The LOCI implementation strategy was designed to improve general and implementation leadership, subsequent implementation climate, and provider implementation behaviors including the adoption and use of EBP with fidelity. The ultimate goal is to improve client engagement in services and patient outcomes. Preliminary results suggest that LOCI can improve the context for implementation or EBPs, and that LOCI can help to focus leaders’ attention on improving implementation.

Trial Registration: NCT03042832


1. Aarons GA, Ehrhart MG, Farahnak LR, Hurlburt MS. Leadership and organizational change for implementation (LOCI): a randomized mixed method pilot study of a leadership and organization development intervention for evidence-based practice implementation. Implement Sci. 2015;10:11

2. Aarons GA, Ehrhart MG, Moullin JC, Torres EM, Green AE. Testing the Leadership and Organizational Change for Implementation (LOCI) intervention in substance abuse treatment: A cluster randomized trial study protocol. Implement Sci. 2017;12:29.

3. Aarons GA, Ehrhart MG, Farahnak LR, Sklar M. Aligning leadership across systems and organizations to develop a strategic climate for evidence-based practice implementation. Annu Rev Public Health. ProShow Gold v2.0.1524 crack serial keygen 35:255-274.

A13 Testing a multi-level implementation strategy for two evidence-based autism interventions

Lauren Brookman-Frazee1,2, Aubyn Stahmer3, Allison Jobin1,2, Kristine Carandang1,2

1Department of Psychiatry, University of California San Diego, San Diego, CA, USA; 2Child and Adolescent Services Research Center, University of California San Diego, San Diego, CA, USA; 3MIND Institute, University of California Davis, Davis, CA, USA

Correspondence: Lauren Brookman-Frazee (


Children with ASD are a high priority population served in multiple public service systems. Evidence-based behavioral interventions are available [1-2], however, they are not routinely delivered in community care [3-6]. In response, our research groups used community-partnered approaches to adapt and test ASD interventions for routine delivery - “AIM HI” in children’s mental health [7] and “CPRT” in education [8]. We identified implementation leadership and climate as key implementation mechanisms in recent community effectiveness trials. We are conducting two, coordinated studies testing the effectiveness of an adapted version of the Leadership and Organizational Change for implementation (LOCI) strategy [9] as part of an implementation package [10]. This presentation describes the application of LOCI in two ASD services contexts for two EBIs.

Materials and Methods

The TEAMS project includes two linked randomized Hybrid Type 3 implementation trials to test two implementation strategies when paired with AIM HI or CPRT and examine mechanisms of these strategies, including implementation leadership and climate. The TEAMS Leadership Institute (TLI) applies LOCI as follows: (1) uses the LOCI components linked to mechanisms identified in the AIM HI and CPRT community effectiveness trials (i.e. implementation leadership and climate modules); (2) targets executive and mid-level leaders required to coordinate implementation; and (3) targets implementation of specific ASD interventions. We present process data on TLI implementation and initial themes from qualitative interviews to examine leader perceptions of the utility of TLI components and their impact on EBI implementation.


To date, TLI has been conducted in 18 programs/districts in three California counties including 18 workshops and 152 coaching calls. Preliminary themes from interviews with 6 leaders who completed TLI indicate that TLI is feasible and useful to (1) convey to staff the importance of systematically planning EBI implementation, and (2) to maintain leader Inovative Logic Crosstrainer II v4.5.7 English by EViDENCE and focus on executing strategic initiatives around AIM HI and CPRT amidst competing demands.


Preliminary data indicate the TEAMS application of LOCI is feasible and perceived as effective in facilitating the implementation of two ASD interventions. Future analyses will examine the impact of LOCI on targeted mechanisms – implementation leadership and climate.

Trial Registration: NCT03380078


1. National Camtasia Studio 2021.0.9 Crack With Keygen [100% Working] Center. National Standards Project, Phase 2. 2015.

2. Wong C, Odom SL, Hume KA, Cox AW, Fettig A, Kucharczyk S, Brock ME, Plavnick JB, Fleury VP, Schultz TR. Evidence-based practices for children, youth, and young adults with autism spectrum disorder: a comprehensive review. J Autism Dev Disord. 2015;45(7):1951–1966.

3. Brookman-Frazee L, Baker-Ericzén M, Stadnick N, Taylor R. Parent perspectives on community mental health services for children with autism spectrum disorders. J Child Fam Stud. 2012;21(4):533–544.

4. Brookman-Frazee L, Drahota A, Stadnick N, Palinkas LA. Therapist perspectives on community mental health services for children with autism spectrum disorders. Adm Policy Ment Health. 2012;39(5):365-373.

5. Brookman-Frazee L, Taylor R, Inovative Logic Crosstrainer II v4.5.7 English by EViDENCE, Garland AF. Characterizing community-based mental health services for children with autism spectrum disorders and disruptive behavior problems. J Autism Dev Disord, Inovative Logic Crosstrainer II v4.5.7 English by EViDENCE. 2010; 40(10):1188–201.

6. Stahmer AC, Collings NM, Palinkas LA. Early intervention practices for children with autism: descriptions from community providers. Focus Autism Other Dev Disabil, 2005;20(2):66-79.

7. Brookman-Frazee L, Roesch S, Chelbowski C, Baker-Ericzen, Ganger W. Effectiveness of training Inovative Logic Crosstrainer II v4.5.7 English by EViDENCE to deliver an individualized mental health intervention for children with ASD in publicly funded mental health services: a cluster randomized clinical trial. JAMA Psychiatry. 2019;76(6):574-583.

8. Suhrheinrich J, Rieth SR, Dickson KS, Roesch S, Stahmer AC. Classroom pivotal response teaching: Teacher training outcomes of a community efficacy trial. Teach Educ Spec Educ. 2019.

9. Aarons GA, Ehrhart MG, Moullin JC, Torres EM, Green AE. Testing the Leadership and Organizational Change for Implementation (LOCI) Intervention in substance abuse treatment: a cluster randomized trial study protocol. Implement Sci. 2017;12(1):29.

10. Brookman-Frazee L. Stahmer AC. Effectiveness of a multi-level implementation strategy for ASD interventions: study protocol for two linked cluster randomized trials. Implement Sci. 2018;13(66).

A14 Translation and adaptation of LOCI for implementation of evidence-based treatment for PTSD in Norwegian child and adult mental health care services

Erlend Høen Laukvik, Ane-Marthe Solheim Inovative Logic Crosstrainer II v4.5.7 English by EViDENCE, Karina M. Egeland

Norwegian Centre for Violence and Traumatic Stress Studies, Oslo, Norway

Correspondence: Erlend Høen Laukvik (


The Norwegian Centre for Violence and Traumatic Stress Studies (NKVTS) is commissioned by the Ministry of Health and Care Services to implement evidence-based treatment for post-traumatic stress disorder (PTSD) in both child and adult mental health care services. As part of this effort, the Leadership and Organizational Change for Implementation (LOCI) [1] was translated and adapted for the implementation of trauma treatment in Norwegian health trusts. The aim of the project is to evaluate the effectiveness of LOCI in supporting the implementation of evidence-based treatment for PTSD in Norwegian specialized mental health clinics [2]. The presentation will identify Tag: zemana antilogger keygen implementation determinants, targets, and mechanisms being examined.

Materials and Methods

The study is a Type III scale-out project [3]. Several a-priori adaptations were made, including translation of the LOCI materials into Norwegian, and tailoring of the LOCI fidelity tool. The study design is a stepped wedge cluster randomized trial with random and sequential enrollment of clinics into three cohorts, with crossover of clusters from control conditions to active intervention conditions based on time intervals. Executives, clinic leaders, and therapists complete surveys assessing leadership and implementation climate at baseline, 4, 8, 12, 16, and 20 months. At baseline, all therapists at the participating clinics were trained in trauma screening and a sub sample in the treatment models for PTSD (TF-CBT, EMDR, CT-PTSD), and units were randomly assigned to one of three cohorts. In addition, the strategy uses the 360 degrees assessments to inform subsequent work on tailored leadership and climate development plans to enhance implementation. Therapy sessions are audio or video recorded and scored for fidelity. Patients complete surveys assessing symptom development during the therapy process.


Consistent with the LOCI theoretical model, assessment of mechanisms will examine the effects of leadership on EBP fidelity through its effect on implementation climate.


This study will provide knowledge about the effect of the LOCI program within a Norwegian context. As such, the results might inform evidence-supported implementation strategies that could help sustain national-wide implementation of evidence-based trauma treatment and increase the quality and effectiveness of Norwegian health services.


1. Aarons GA, Ehrhart MG, Moullin JC, Torres EM, Green AE. Testing the Leadership and Organizational Change for Implementation (LOCI) Intervention in substance abuse treatment: a cluster randomized trial study protocol. Implement Sci. 2017;12(1):29.

2. Egeland KM, Skar AMS, Endsjø M, Laukvik EH, Bækkelund H, Babaii A, Granly LB, Husebø GK, Borge RH, Ehrhart MG, Sklar M. Testing the leadership and organizational change for implementation (LOCI) intervention in Norwegian mental health clinics: a stepped-wedge cluster randomized design study protocol. Implement Sci. 2019;14(1):28.

3. Aarons GA, Sklar M, Mustanski B, Benbow N, Brown CH. “Scaling-out” evidence-based interventions to new populations or new health care delivery systems. Implement Sci. 2017;12(1):111.

A15 Making sense of context: a systematic review

Lisa Rogers, Aoife DeBrún, Eilish Inovative Logic Crosstrainer II v4.5.7 English by EViDENCE of Nursing, Midwifery and Health Systems, University College Dublin, Dublin, Ireland

Correspondence: Lisa Rogers (


The uptake of evidence-based healthcare interventions is challenging, with, on average, a 17-year time gap between the generation of evidence and implementation of interventions into routine practice [1]. Although contextual factors such as culture are strong influences for successful implementation [2], context remains a poorly understood construct, with a lack of consensus regarding how it should be defined and accounted for within research [3]. A systematic review was conducted to address this issue by providing an insight into how context is defined and assessed within healthcare implementation science literature and develops a definition to better enable effective measurement of context.

Materials and Methods

The databases of PubMed, PsycInfo, CINAHL, and EMBASE were searched. English language empirical studies published in the previous 10 years were included if context was treated as a key component in implementing a healthcare initiative. Articles also needed to provide a definition and measure of context in order to be included. Results were synthesised using a narrative approach and supported using PRISMA guidelines for the conduct and reporting of systematic reviews.


The searches yielded 3,021 records of which 64 met the eligibility criteria and were included. Studies used a variety of definitions. Some listed contextual factors (n=19) while others documented sub-elements of a framework that included context (n=19). Remaining articles provided a rich definition of an aspect of context (n=14) or context generally (n=12), Inovative Logic Crosstrainer II v4.5.7 English by EViDENCE. Quantitative studies mostly employed the Alberta Context Tool while qualitative papers used a Inovative Logic Crosstrainer II v4.5.7 English by EViDENCE of frameworks with Promoting Action on Research Implementation in Health Services framework the most highly cited. Mixed methods studies used diverse approaches to assess context. Some used frameworks to inform the methods chosen while others used quantitative measures to inform qualitative data collection. Most papers (n=50) applied the chosen measure to all aspects of study design with a majority analysing context at an individual or level (n=51).


This review highlighted inconsistencies in defining and measuring context which supported the development of an enhanced understanding for this construct. By providing this consensus, improvements in implementation processes may result as greater understanding will help researchers appropriately account for context in research.


1. Bauer MS, Damschroder L, Hagedorn H, Smith J, Kilbourne AM. An introduction to implementation science for the non-specialist. BMC Psychol. 2015;3(1). doi:10.1186/s40359-015-0089-9

2. Proctor EK, Powell BJ, Baumann AA, Hamilton AM, Santens RL. Writing implementation research grant proposals: ten key ingredients. Implement Sci. 2012;7(96). Accessed May 15, 2018.

3. Nilsen P. Making sense of implementation theories, models, and frameworks. Implement Sci. 2015;10(1):53. doi:10.1186/s13012-015-0242-0

A16 Implementing mental health assessment in a juvenile detention behavioral health unit: lessons learned from a community academic partnership

Brittany Rudd1, Jacquelyn George2, Lauren Cliggitt3, Sean Snyder4, Mynesha Whyte4, Rinad S. Beidas1

1Department of Psychiatry, University of Pennsylvania, Philadelphia, PA, USA; 2College of Public Health, Temple University, Philadelphia, PA, Inovative Logic Crosstrainer II v4.5.7 English by EViDENCE, USA; 3Community Behavioral Health, Philadelphia, PA, USA; 4Hall Mercer Community Mental Health Center, University of Pennsylvania, Philadelphia, PA, USA

Correspondence: Brittany Rudd (


The dual objectives of juvenile justice are to assure youth safety while in custody and to facilitate rehabilitation, Inovative Logic Crosstrainer II v4.5.7 English by EViDENCE. Suicide is the second leading cause of death among 10-25 year olds [1], and is four times more likely among youth who enter juvenile justice (JJ) settings [2]. As youth in juvenile detention are at risk for Inovative Logic Crosstrainer II v4.5.7 English by EViDENCE in suicidal behaviors, it is critical that behavioral health clinicians in juvenile detention settings conduct systematic evidence-based suicide risk, as well as general mental health, assessment. Recommendations for assessment in juvenile detention exist [3], but there is little guidance regarding how to implement them among behavioral health clinicians. The current presentation will describe a community academic (CAP) partnership, and the process that the partners underwent to implement a systemic protocol for assessing youth in a juvenile detention behavioral health unit.

Materials and Methods

A CAP was developed and a quality improvement procedure was utilized to develop and implement the assessment protocol.


The CAP team included the service clinicians (Snyder and Whyte), and clinical supervisor (Cliggitt) in a behavioral health unit housed in a large, juvenile detention center in an urban city in Pennsylvania, as well as researchers from the University of Pennsylvania (Rudd, George, and Beidas). The development of the assessment protocol was an iterative process that occurred over eight months. The process started with a comprehensive review of current workflow and workflow infrastructure, including how youth were referred to the behavioral health unit and the information behavioral health unit staff had about youth prior to their intake. Iterative changes to workflow procedures were needed, including developing infrastructure to support assessment (e.g., developing report templates) during the behavioral health intake appointment. Finally, several assessment measures were piloted to determine fit.


The creation of a CAP was key to developing and implementing a comprehensive and feasible mental health and suicide assessment protocol. Lessons learned from the application of implementation science to the juvenile detention context from the joint perspectives of researcher (Rudd) and clinician (Synder) stakeholder perspectives will be presented.


1. Heron MP. Deaths: Leading causes for 2016. National Vital statistics reports. 2018;67(6).

2. Wasserman GA, McReynolds LS. Suicide risk at juvenile justice intake. Suicide Life Threat Behav. 2006;36(2):239-249.

3. Grisso T, Underwood LA. Screening and assessing mental health and substance use disorders among youth in the juvenile justice system. a resource guide for practitioners. US Department of Justice. 2004. /ojjdp/204956.pdf.

A17 DIY implementation: lessons from a practitioner-led implementation of an evidence-based practice

Sean Wright1, Sonia Combs2

1Lutheran Community Services Northwest, Spokane, WA, USA; 2Cor Counseling and Wellness, Spokane, WA, USA

Correspondence: Sean Wright (


Reports of implementation efforts initiated at Inovative Logic Crosstrainer II v4.5.7 English by EViDENCE practitioner level are uncommon. To address this gap, we describe the results of and lessons from an ongoing practitioner-led implementation of Acceptance and Commitment Therapy (ACT), an evidence-based practice, in a community mental health center team.

Materials and Methods

We used a variety of implementation strategies (mostly training) during an ongoing implementation of ACT. Initially, we conducted a mixed methods study of the facilitators and barriers to implementation, collecting qualitative and quantitative survey data anonymously at two time points, sampled from all clinical staff (N=39) at our agency. The survey measured attitudes, knowledge, experience, and acceptability of the EBP. We assessed the significance of changes in Likert ratings using the sign test [1]. We used thematic analysis to code qualitative data. Recently, penetration was measured by relative use of ACT in a one-month sample of progress notes and by the relative percentage of team members using ACT. Implementation strategies used were identified by retrospective review and coded in accordance with the Expert Recommendations for Implementing Change (ERIC) project [2] and classified into concept mapping clusters for ERIC strategies [3]. We created a timeline of implementation activities and identified key individuals who facilitated these activities.


15 pairs of pre-post survey measures indicated that initial training was associated with increases in identification as an ACT therapist (Z=-2.12,p=0.035), perceived ability to demonstrate ACT (Z=-3.00,p=0.002), and a trend toward increased use of ACT (Z=-1.90,p=0.055). Qualitative analyses were consistent with the existing literature on facilitators and barriers to EBP adoption in community mental health. ACT use in a recent one-month window was evidenced with 7.9% of progress notes documenting use of ACT (baseline before implementation: 0%) and 32% of eligible clinicians documenting ACT use in progress notes (initial baseline: 0%). Evidence for use of 21 of the 73 ERIC implementation strategies was documented. The strategies are distributed across all 9 concept mapping clusters, with the Train and Educate Stakeholders cluster most represented (5 of 11 strategies). Three key individuals were identified.


Practitioner-led implementation is feasible. Implementation strategies can inform practitioner efforts.


1. Roberson PK, Shema SJ, Mundfrom DJ, Holmes TM. Analysis of paired Likert data: how to evaluate change and preference questions. Fam Med. 1995;27(10):671-675.

2. Powell BJ, Waltz TJ, Chinman MJ, Inovative Logic Crosstrainer II v4.5.7 English by EViDENCE, Damschroder LJ, Smith JL, Matthieu MM, Proctor EK, Kirchner JE. A refined complication of implementation strategies: results from the Expert Recommendations for Implementing Change (ERIC) project. Implement Sci. 2015;10:21.

3. Waltz TJ, Powell BJ, Matthieu MM, Damschroder LJ, Inovative Logic Crosstrainer II v4.5.7 English by EViDENCE, Chinman MJ, Smith JL, Proctor EK, Kirchner JE. Use of concept Inovative Logic Crosstrainer II v4.5.7 English by EViDENCE to characterize relationships among implementation strategies and assess their feasibility and importance: results from the Expert Recommendations for Implementing Change (ERIC) study. Implement Sci. 2015;10:109.

A18 Implementation of an educator participatory program for improving work environments on health and wellbeing: a mixed methods approach

Lisa Sanetti, Alexandra Pierce, Michele Femc-Bagwell, Alicia Dugan

Neag School of Education University of Connecticut, Storrs, CT, USA

Correspondence: Lisa Sanetti (


A chronic and increasing challenge to employee wellness in schools is teacher stress [1]. Teachers are tied with nurses as having the highest rates of daily stress among occupations [2]. Chronic high levels of teacher stress are associated with (a) increased rates of physical and psychological health problems, including anxiety, depression, cardiovascular disease, and poor sleep quality; (b) poor job performance, including absenteeism, negative interactions, poor relationships with students, and poor classroom management, and (c) poor student outcomes, including low rates of academic achievement, lower levels of social adjustment, and increased rates of problem behavior [1,3]. Further, chronic teacher stress is the primary factor associated with Inovative Logic Crosstrainer II v4.5.7 English by EViDENCE high rate of teachers leaving the profession for reasons Total Security 1 User 3 Yrs crack serial keygen than retirement, which has nearly doubled over the past 25 years, constituting the primary cause of teacher shortages nationwide [4]. A critical need to address teacher health and wellbeing exists, yet, on average, only 31.4% of schools offer workplace health and wellness promotion programs; most of these programs are top-down, one-size-fits-all approaches that are either ineffective or unsustainable [5].

Materials and methods

The purpose of this mixed methods study was to implement the Healthy Workplace Participatory Program (HWPP), an evidence-based approach that engages front-line employees (i.e., teachers) and supervisors (e.g., administrators) in a collaborative, iterative design of workplace health and wellness interventions [6].


This participatory approach allows for (a) identification of health and wellness issues most salient to employees; (b) development of a wider range of interventions as employees are more aware of complex interactions between their work organization, workplace, and lifestyle; and (c) identification of potential intervention barriers and facilitators; (d) increased buy-in to problem definition and intervention design; and (e) establishment of a supportive organizational culture and processes for a self-correcting and sustainable health and wellness promotion program. The HWPP has been shown to effectively increased employee health and wellbeing in a wide range of worksites [6]; this is the first implementation effort in schools.


Results of focus groups as well as formative and summative data related to implementation and intervention processes, strategies, and outcomes across EPIS phases in two 3rd-5th grade elementary schools in the Northeast will be presented.


1. Flook L, Goldberg S, Pinger L, Bonus K, Davidson R. Mindfulness for teachers: a pilot study to assess effects on stress, burnout, and teaching efficacy. Mind Brain Educ. 2013;7(3):182-195.

2. Gallup. State of America’s Schools 2014. Accessed 22 July 2018.

3. Roeser RW, Skinner E, Beers J, Jennings PA. Mindfulness training and teachers’ professional development: an emerging area of research and practice. Child Dev Perspect. 2012;6(2):167-173.

4. Goldring R, Taie S, Riddles M. Teacher attrition and mobility: results from the 2012–13 teacher follow-up survey (NCES 2014-077). 2014. Accessed 3 April 2016.

5. Naghieh A, Montgomery P, Bonell CP, Thompson M, Aber JL. Organisational interventions for improving wellbeing and reducing work-related stress in teachers. Cochrane Database Syst Rev. 2015;4:CD010306.

6. Robertson M, Henning R, Warren N, Dove-Steinkamp M, Tibirica L, Bizarro A, CPH-NEW Research Team. The intervention design and analysis scorecard: A planning tool for participatory design of integrated health and safety interventions in the workplace. J Occup Environ Djay pro serial key Archives. 2013;55:S86-S88.

A19 Assessing intermediary organization capacity for active implementation support: development and collaborative early usability appraisal of an intermediary organization capacity assessment tool

Robin Jenkins, William Aldridge, Rebecca Roppolo

Frank Porter Graham Child Development Institute, University of North Carolina at Chapel Hill, Chapel Hill, NC, USA

Correspondence: Robin Jenkins (


Many evidence-based practices (EBPs) rely on multiple dissemination supports to assist scaling to achieve population benefits. Intermediary organizations (IOs) are often key Inovative Logic Crosstrainer II v4.5.7 English by EViDENCE leveraging critical functions in the overall support system to enhance diffusion strategies [1-3]. Despite the prevalence of intermediaries as important accelerators of evidence-based practices, little is known about which IO strategies are most effective in ensuring implementation and scaling success or how strategies link to existing IO capacities [3]. Further, there is a dearth of information regarding IO capacity assessments relative to their capabilities to effectively diffuse EBP’s or to perform active implementation support.

Materials and Methods

Working with a public-private partnership (state government agencies and private funders) to scale Triple P statewide in North and South Carolina, both states have selected IOs to enhance statewide Triple P implementation. To assess IO capacity for planning and delivering implementation supports, a tool was needed to establish baseline capacity and to guide support planning. The Intermediary Organization Capacity Assessment (IOCA) was developed as an IO capacity assessment tool aligned with Mettrick et al.’s five observed functions.


Early capacity assessment data and collaborative qualitative usability feedback from partners indicate that the tool is demonstrating practical utility toward capacity assessments and planning for ongoing support. The IOCA appears to align well with Mettrick et al.’s functional groups of IO support activities. Early feedback suggests that use of the tool also aids in transferring knowledge of implementation science-informed strategies to IO partners in functionally informative, practical ways.


The IOCA is demonstrating good alignment with known classes of IO support functions. It is also providing practical usability relative to understanding baseline IO capacity to deliver ongoing supports for scaling of EBPs. IOs that have experience with the tool report improved understanding of implementation science-informed strategies and tools that can better guide them in their support activities.


1. Franks RP, Bory CT. Who supports the successful implementation and sustainability of evidence-based practices? Defining and understanding the roles of intermediary and purveyor organizations. New Dir Child Adolesc Dev. 2015; 149:41-56. doi:10.1002/cad.20112

2. Mettrick J, Harburger DS, Kanary PJ., Lieman RB, Zabel,M. Building cross-system implementation centers: a roadmap for state and local child serving agencies in developing Centers of Excellence (COE). Baltimore, MD: The Institute for Innovation & Implementation, University of Maryland. 2015.

3. Proctor E, Hooley C, Morse A, McCrary S, Kim H, Kohl PL. Intermediary/purveyor organizations for evidence-based interventions avid pro tools crack Archives the US child mental health: characteristics and implementation strategies. Implement Sci. 2019;14(1):3. doi:10.1186/s13012-018-0845-3

A20 A tailored implementation approach to improving PTSD care in military treatment facilities: integrating practice-based knowledge and implementation science

David Riggs1, Katherine Dondanville2, Elisa Borah3, Craig Rosen4

1Center for Deployment Psychology, Uniformed Services University, Bethesda, MD, USA; 2Department of Psychiatry and Behavioral Sciences, University of Texas Health Science Center at San Antonio, San Antonio, TX, USA; 3Department of Psychiatry, University of Texas at Austin, Austin, TX, USA; 4National Center for PTSD, VA Palo Alto Health Care System, Palo Alto, CA, USA

Correspondence: David Riggs (


The panel will discuss integration of practical lessons learned from clinicians and administrators with principles of implementation science to develop a program to increase use of evidence-based psychotherapy (EBP) for PTSD in military treatment facilities (MTFs). Despite efforts to train military providers in EBPs, only a minority of service members receive them [1]. Implementation barriers likely vary across MTFs, which differ in size, resources, command structure, and implementation climate. Increased use of EBPs likely requires a tailored approach that aligns implementation strategies to local conditions [2].

Materials and Methods

The Targeted Assessment and Context-Tailored Implementation of Change Strategies (TACTICS) program combines needs assessment, a rubric for aligning implementation strategies to local barriers and facilitators, and external facilitation to help clinics enact a collaboratively developed Pinnacle Studio Crack With Keygen 2021 Download {Win/Mac} plan. Through experience working with MTFs, the Center for Deployment Psychology (intermediaries) identified common implementation barriers and potential context-specific strategies to address them. These were augmented with additional relevant strategies from the Expert Recommendations for Implementing Change project [3] and input from experienced implementers. Barriers and facilitators in the resulting TACTICS rubric were then mapped backed to domains of the Consolidated Framework for Implementation Research [4].

After getting leadership approval and identifying a site champion, the five-month TACTICS process involves conducting needs assessment interviews with relevant staff and reviewing clinic data to identify barriers and facilitators, using the TACTICS rubric to identify potential change targets and strategies to address local conditions, and meeting with staff to develop the implementation plan. This is followed by weekly coaching calls (external facilitation) to support the champion in enacting changes to increase use of evidence-based psychotherapy.


TACTICS rubric development is completed and is Inovative Logic Crosstrainer II v4.5.7 English by EViDENCE pilot tested at one site. After this development phase, TACTICS will be tested in a stepped-wedge randomized trial in eight military treatment facilities.


Development of the TACTICS program was informed by intermediaries’ practical knowledge from military clinicians, implementation experience, and by implementation science frameworks. If successful, TACTICS provides a barrier-to-solution tailoring framework informed by implementation practitioners, researchers, and local staff.


1. Hepner KA, Roth CP, Sloss EM, Paddock SM, Iyiewuare PO, Timmer MJ, Pincus HA. Quality of care for PTSD and depression in the military health system: Final report. RAND Health Q. 2018;7(3):3.

2. Powell BJ, Beidas RS, Lewis CC, Aarons GA, McMillen JC, Proctor EK, Mandell DS. Methods to improve the selection and tailoring of implementation strategies. J Behav Health Serv Res. 2017;44(2):177-194. doi:10.1007/s11414-015-9475-6.

3. Powell BJ, Waltz TJ, Chinman MJ, Damschroder LJ, Smith JL, Matthieu MM, Proctor EK, Kirchner JE. A refined compilation of implementation strategies: results from the Expert Recommendations for Implementing Change (ERIC) project. Implement Sci. 2015;10:21. doi:10.1186/s13012-015-0209-1.

4. Damschroder LJ, Aron DC, Keith RE, Kirsh SR, Alexander JA, Lowery, JC. Fostering implementation of health services research findings into practice: a consolidated framework for advancing implementation science. Implement Sci. 2009; 4:50. doi:10.1186/1748-5908-4-50.

A21 Rubber meets the road: how one intermediary organization uses implementation science to inform training and implementation supports for a large state system of behavioral health

Sapana Patel1,2, Lisa Dixon1,2

1Department of Psychiatry, Columbia University, New York, NY, USA; 2The New York State Psychiatric Institute, New York, NY, USA

Correspondence: Sapana Patel (


At federal, state, and local levels, stakeholders are focused on developing, disseminating, and implementing evidence-based practices (EBPs). Intermediary organizations are entities that help agencies or systems develop, implement, and sustain evidence-based practices [1]. Little is known about how implementation science frameworks, strategies and tools that are used by intermediary organizations charged with scaling evidence-based practices for a large state system of behavioral health.

Materials and Methods

The Center for Practice Innovations (CPI), at Columbia Psychiatry and the New York State Psychiatric Institute, is an intermediary organization whose mission is to support the New York State Office of Mental Health in the use of EBPs throughout community-based mental health agencies in New York State. CPI’s role includes: (a) public awareness, and education; (b) scalable dissemination of training in EBPs; (c) implementation support through learning collaboratives; (d) quality improvement; and (e) outcome evaluation. We will describe empirical approaches to the development, dissemination of scalable training and implementation support for a range of initiatives at CPI.


Grounded in the Consolidated Framework for Implementation Research [2], we will present on the CPI practice change model and how the CFIR assists in planning for post-training implementation support and the identification of barriers and facilitators to implementation. Using Advanced SystemCare Pro 14.6.0 Crack is here! published taxonomy, Expert Recommendations for Implementing Change [3], we will describe a range of implementation strategies (e.g., instructional design methods, user-centered design, stakeholder engagement) that inform the development of scalable online training and identify targets for post-training implementation activities. Lastly, we will provide examples of online training evaluation [4] and challenges faced in reporting on the impact of implementation strategies [5-6] within a large system of behavioral healthcare.


Although a balancing act, it is possible for intermediary organizations to remain flexible, Inovative Logic Crosstrainer II v4.5.7 English by EViDENCE, efficient, and rapid in response to the mission of real-world dissemination and implementation of EBPs and use empirically-driven distance and E-learning and implementation science approaches. There are opportunities for mutual learning, synergy and collaboration to advance the field of implementation science for researchers and practitioners.


1. Franks RP, Bory CT. Who supports the successful implementation and sustainability of evidence-based practices? Defining and understanding the roles of intermediary and purveyor organizations. New Dir Child Adolesc Dev. 2015; 149:41-56. doi:10.1002/cad.20112

2. Damschroder LJ, Aron DC, Keith RE, Kirsh SR, Alexander JA, Lowery JC. Fostering implementation of health services research findings into practice: a consolidated framework for advancing implementation science. Implement Sci. 2009; 4:50.

3. Powell BJ, Waltz TJ, Chinman MJ, Damschroder LJ, Smith JL, Matthieu MM, Proctor EK, Kirchner JE. A refined compilation of implementation strategies: results from the Expert Recommendations for Implementing Change (ERIC) project. Implement Sci. 2015;10:21.

4. Kirkpatrick DKJ, Kirkpatrick JD. Evaluating training programs: the four levels (3rd Edition). San Francisco, CA: Berrett-koehler Publishers; 2019.

5. Proctor EK, Powell BJ, McMillen JC. Implementation strategies: recommendations for specifying and reporting. Implement Sci. 2013; 8:139.

6. Gold R, Bunce AE, Cohen DJ, Hollombe C, Nelson CA, Proctor EK, Pope JA, DeVoe JE. Reporting on the strategies needed to implement proven interventions: an example from a “real-world” cross-setting implementation study. Mayo Clin Proc. 2016;91(8):1074-83.

A22 Utilization of train-the-trainer programs to support the sustainability of evidence-based trauma-informed interventions: the perspectives of model developers, trainers, and intermediary agencies within the National Child Traumatic Stress Network

Shannon Chaplo1, George Ake1,2, Lisa Amaya-Jackson1,2, Byron J. Powell3, IDM Crack 6.35 Build 9 Retail With Serial Key [Newest] Sprang4

1Department of Psychiatry and Behavioral Science, Duke University Medical Center, Durham, NC, USA; 2National Center for Child Traumatic Stress, Washington, DC, USA; 3Brown School, Washington University in St. Louis, St. Louis, MO, USA; 4Department of Psychiatry, University of Kentucky, Lexington, KY, USA

Correspondence: Shannon Chaplo (


Train-the-Trainer programs (TTTs) refer to “a program or Inovative Logic Crosstrainer II v4.5.7 English by EViDENCE where individuals in a specific field receive training in a given subject and instruction on how to train, monitor, and supervise other individuals in the approach [1].” TTTs are implementation strategies intended to increase the reach and sustainment of evidence-based interventions in mental health agencies, and address other challenges such as therapist attrition and developer succession planning [2-3]. Several trauma-informed interventions for children have TTTs; however, there is no standardized protocol for developing or delivering TTTs. As the need to disseminate and sustain trauma-informed interventions grows, the need to develop Inovative Logic Crosstrainer II v4.5.7 English by EViDENCE for TTTs becomes imperative. The objective of this project is to better understand the state of TTTs for trauma-informed interventions utilized by members and consumers of the National Child Traumatic Stress Network (NCTSN).

Materials and Method

Duke University study staff partnered with members of the NCTSN Implementation Advisory Committee to develop a survey exploring TTTs in the NCTSN. The survey was designed to gather the perspective of developers of treatments, practices, and curricula; professionals that become trainers through TTTs; and agency training directors that serve as consumers of TTTs. Developers will answer a series of questions about the development and implementation of their TTT program. Trainers and agency directors will be asked about their experience participating in a TTT program. All respondents will be asked about the components of their TTTs, barriers to using or developing TTTs, and facilitators of developing or using TTTs.


No results are currently available. The study has secured IRB approval and the survey will launch in April 2019. Results will be analyzed in summer 2019.


Surveying each of these audiences will help us to better understand the varying components of TTTs, and barriers and facilitators of their use within the NCTSN. We plan to use the survey results for training purposes and resource development to enhance the use of TTTs within the NCTSN (and in other relevant settings) to implement and sustain trauma-informed interventions.


1. Pearce J, Mann, MK, Jones, C, van Buschbach, S, Olff, M, Bisson, JI. The most effective way of delivering a Train‐the‐Trainers program: a systematic review. J Contin Educ Health Prof. 2012; 32:215-226.

2. Bero L, Grilli R, Grimshaw J, Harvey E, Oxman A, Thomson M. Closing the gap between research and practice: an overview of systematic reviews of interventions to promote implementation of research findings by health care professionals. BMJ. 1998;317:465–468.

3. Yarber L, Brownson CA, Jacob RR, Baker EA, Jones E, Baumann C, Deshpande AD, Gillespie KN, Scharff DP, Brownson RC. Evaluating a train-the-trainer approach for improving capacity for evidence-based decision making in public health. BMC Health Serv Res. 2015;15:547.

A23 Nudge yourself: stakeholder design of implementation strategies that leverage insights from behavioral economics

Briana S. Last, Courtney Benjamin Wolk, Rinad S. Beidas

Department of Psychiatry, University of Pennsylvania, Philadelphia, PA, USA

Correspondence: Briana S. Last (


Though several evidence-based practices (EBPs) exist for depression, only a fraction of individuals receive treatment. One major challenge to treatment is the identification of individuals in need of care. In response to this need, health care systems such as the University of Pennsylvania (Penn) have mandated universal screening of depression in primary care settings based on evidence. However, adherence to the mandate at Penn is much lower than anticipated (only 40% of eligible patients are screened). In our study, we partnered with front line clinicians and staff to increase depression screening at Penn using innovative approaches from implementation science and behavioral economics.

Materials and Methods

This project will engage in a participatory process with key stakeholders to design implementation strategies to increase universal depression screening in primary care. In particular, we fill focus on designing a subset of implementation strategies—nudges, as they are called in behavioral economics [1]—that alter the choice architecture, or the way options are presented to optimize choices. First, we began by conducting an innovation tournament, a crowdsourcing technique [2], with physicians, nurses, medical assistants, behavioral health clinicians, and front-desk stuff currently involved in administering the electronic depression screener, the two-item Patient Health Questionnaire (PHQ-2) at their practices. The tournament will generate ideas on how to increase the implementation of the PHQ-2. Next, we will fine-tune the ideas generated from the innovation tournament with a team of stakeholders, behavioral economists, and implementation scientists using behavioral economic theory [3]. The product of this project will be a toolkit of implementation strategies, that are theoretically motivated and acceptable to a range of relevant stakeholders, a subset of which will be later refined through a more rigorous piloting process.


The innovation tournament closes mid-April, 2019. Ideas from the tournament will be refined into a toolkit of implementation strategies by September 2019.


Our study responds to the need for interdisciplinary, theoretically informed, and participatory approaches to designing implementation strategies. The results from our work will shed light on whether these approaches show promise.


1. Thaler RH, Sunstein CR. Nudge: Improving decisions about health, wealth, and happiness. Revised and Expanded Edition. New York, NY: Penguin Books; 2009.

2. Terwiesch C, Mehta SJ, Volpp KG. Innovating in health delivery: the Penn medicine innovation tournament. Healthc. 2013;1(1-2):37–41.

3, Inovative Logic Crosstrainer II v4.5.7 English by EViDENCE. Eldredge LKB, Markham CM, Ruiter RA, Kok G, Fernandez ME, Parcel GS. Planning health promotion programs: an intervention mapping approach. San Francisco: John Wiley & Sons; 2016.

A24 Using stakeholder values to promote implementation of an evidence-based mHealth intervention for addiction treatment in primary care

Correspondence: Andrew Quanbeck (

College of Engineering, University of Wisconsin – Madison, Madison, WI, USA


The majority of evidence-based practices do not find their way into clinical use, including mobile health (mHealth) technologies. This presentation describes a novel decision-framing model that gives implementers a method for eliciting the perceived gains and losses that different stakeholder groups trade off when faced with the decision of whether to adopt an evidence-based mHealth intervention.

Materials and Methods

The decision-framing model integrates insights from behavioral economics [1,2] and game theory [3]. The approach was applied retrospectively in a parent implementation research trial that introduced an mHealth system to 268 patients in three U.S. clinics offering primary and behavioral healthcare services. The mHealth system, called Seva, supports patients with addiction. Individual and group interviews were conducted to elicit stakeholder considerations from 23 clinic staff members and 6 patients who were involved in implementing Seva. Considerations were used to construct “decision frames” that trade off the perceived value of adopting Seva vs. maintaining the status quo from each stakeholder group’s perspective. The face validity of the decision-framing model was assessed by soliciting feedback from the stakeholders whose input was used to build it.


Primary implementation considerations were identified for each stakeholder group. Clinic managers perceived the greatest potential gain to be providing better care for patients, and the greatest potential loss to be cost, expressed in terms of staff time, sustainability, and opportunity cost. All clinical staff considered time their foremost consideration—primarily in negative terms (e.g., cognitive burden associated with learning a new system) but potentially positively (e.g., if Seva could automate functions done manually). Patients considered safety (anonymity, privacy, and coming from a trusted source) to be paramount. When considerations were compiled into decision frames that traded off the gains and losses associated with adopting Seva, only one stakeholder group—patients—expressed a positive overall value, and these were the stakeholders who used Seva most.


This paper presents a systematic method of inquiry to elicit stakeholders’ considerations when deciding to adopt a new technology. Stakeholder considerations may be used to adapt mHealth interventions and tailor implementation, potentially increasing the likelihood of implementation success for evidence-based practices and technologies.


1. Tversky A, Kahneman D. The framing of decisions and the psychology of choice. Science 1981; 211(4481):453-8.

2. Kahneman D, Lovallo D, Sibony O. Before you make that big decision… Harv Bus Rev 2011;89(6):50-60.

3. Myerson RB. Game theory: analysis of conflict. Cambridge: Harvard University Press; 1991.

A25 Applying insights from participatory design to design implementation strategies

Rinad S. Beidas1, Nathaniel Williams2, Rebecca Stewart1

1Department of Psychiatry, University of Pennsylvania, Philadelphia, PA, USA; 2School of Social Work, Boise State University, Boise, ID, USA

Correspondence: Rinad S. Beidas (


Public behavioral health systems have increasingly invested in the implementation of evidence-based practices (EBPs), including Philadelphia’s Department of Behavioral Health. Training and technical assistance continue to be the most commonly used strategies to increase use of EBPs, despite findings that organizational barriers matter. Few organizational implementation strategies exist and little is known about how to best design organizational strategies to increase implementation of EBPs using participatory design approaches. We partnered with front line clinicians to develop organizational implementation strategies to improve EBP implementation in community mental health clinics.

Materials and Methods

We engaged in a three-step process to design organizational implementation strategies. First, we launched an innovation tournament to engage clinicians employed within the Philadelphia public behavioral health system to crowd-source how their organizations can support them to use EBPs. We held a community-facing event during which the 6 clinicians who submitted winning ideas presented their ideas to 85 attendees representing a range of stakeholders. Second, we worked with behavioral scientists to refine the ideas to optimize their effectiveness. Third, we launched a system-wide survey targeting approximately 300 stakeholders to elicit preferences for the clinician generated organizational implementation strategies.


We report on the outcomes of the innovation tournament and system-wide survey. A total of 65 ideas were submitted in the innovation tournament by 55 participants representing 38 organizations. The most common categories of ideas pertained to training (42%), compensation (26%), clinician support tools (22%), and EBP-focused supervision (17%). Using an innovation tournament to generate ideas for implementation strategies was feasible and acceptable as demonstrated by the high levels of engagement. However, we also identified barriers (e.g., ensuring that the stakeholder voice was adequately represented throughout all stages). The system-wide survey will be launched in March, 2019; and will close April, 2019.


The approach that we took in designing implementation strategies is promising. Research is needed to test whether strategies developed via these methods are more effective than strategies developed through competing approaches.

A26 Leveraging normative pressure to increase data Inovative Logic Crosstrainer II v4.5.7 English by EViDENCE among therapists working with children with autism

David S. Mandell, Heather Nuske, Emily Becker-Haimes

Department of Psychiatry, University of Pennsylvania, Philadelphia, PA, USA

Correspondence: Emily Becker-Haimes (


Evidence-based practices for children with autism generally follow the principles of applied behavior analysis, which require frequent, systematic data collection. In Philadelphia, as in many systems, children with autism often are accompanied by a one-to-one aide, who is responsible for collecting these data as part of implementing a treatment plan. Direct observations and interviews with these aides and their supervisors confirm that aides rarely collect data in a rigorous manner. These aides often work in isolation and rarely receive consistent supervision, which may lead to the perception that data collection is not expected of them, nor do people in their position collect data in this manner, Inovative Logic Crosstrainer II v4.5.7 English by EViDENCE. We use participatory methods combined with innovative methods borrowed from industry that incorporate the principles of behavioral economics to design implementation strategies to increase aides’ data collection.

Materials and Methods

We partnered with five community agencies and used time-and-motion study methods, an observational technique drawn from scientific management, to understand how one-to-one aides collect data. This process involved querying aides about the decisions they made regarding data collection in the moment. We used participatory design strategies, including an innovation tournament—a method to crowdsource strategy ideas from stakeholders—and a rapid-cycle approach—a method that involves iterative testing Inovative Logic Crosstrainer II v4.5.7 English by EViDENCE refining implementation strategies—to increase one-to-one aides’ data collection. We applied theoretical principles from behavioral economics to refine the implementation strategies generated from the innovation tournament and to test them using our rapid cycling approach.


Data collection is ongoing. Here we present on the time-and-motion studies and results of our innovation tournament. We provide a framework for the rapid-cycle process that is ongoing at the time of this presentation.


This method of data collection in the service of identifying implementation strategies and rapidly testing them holds promise.

A27 Applications of standardized patient methodology to measure fidelity in an implementation trial of the teen marijuana check-up

Bryan Hartzler, Denise Walker, Aaron Lyon, Kevin King, Lauren Matthews, Tara Ogilvie, Devon Bushnell, Katie Wicklander

University of Washington, Seattle, WA, USA

Correspondence: Bryan Hartzler (


A cornerstone of medical education, standardized patients (SPs) are increasingly incorporated in implementation trials Adobe Photoshop 2021 22.5 Crack Mac/Windows FREE Download behavior therapies as a highly valid, advantageous approach to fidelity measurement [1]. Such methodological benefits extend to SP involvement in Inovative Logic Crosstrainer II v4.5.7 English by EViDENCE rehearsal activities often included to support therapy training processes [2]. An ongoing implementation trial examining the Teen Marijuana Check-Up (TMCU) [3], a school-based adaptation of motivational enhancement therapy, incorporates SPs for both purposes [4].

Materials and Methods

In this trial, a set of SPs portray marijuana-using adolescent characters in dyadic interactions with participating school-based staff. As components of TMCU training, two SP-involved training cases—each offering consequence-free opportunities for staff to receive performance-based trainer feedback—supplemented an initial workshop. As components Inovative Logic Crosstrainer II v4.5.7 English by EViDENCE pre- and post-training outcome assessments, two more SP interactions provided behavioral outcome measures. All four SP interactions involved travel to staff workplaces to record a simulated TMCU session, later scored for the following fidelity indices: ratio of reflective listening statements to questions (R:Q), percentage of ‘open-ended’ questions (%OQ), and percentage of ‘complex’ reflective listening statements (%CR).


Recruited from seven high schools, twenty staff completed all four SP interactions. Pre-training SP interactions revealed variable staff performances, with two staff members achieving a TMCU proficiency standard by exceeding benchmarks for all three fidelity indices. In SP-involved training cases, this proficiency standard was achieved by eight staff in an initial case, and by six staff in a more challenging latter case. In eventual post-training SP interactions, five staff met the TMCU proficiency standard. As for mean training Inovative Logic Crosstrainer II v4.5.7 English by EViDENCE, Cohen’s d effect sizes suggest small-to-medium effects on R:Q (d=.20), %CR (d=.36), and %OQ (d=.43), and documented expected needs for subsequent support of TMCU implementation via purveyor coaching as a targeted form of post-training technical assistance.


This trial—wherein SP methodology further extends to monitoring of TMCU fidelity in biannual assessments for two years after training—includes SP roles in outcome assessment and training. Fidelity data from the collective SP interactions evidence sensitivity to hypothesized changes in staff learning, further supporting the use of SPs as means to measure and monitor fidelity in trials examining behavior therapy implementation.

Trial Registration: NCT03111667


1. Imel ZE, Baldwin SA, Baer JS, Hartzler B, Dunn C, Rosengren DB, Atkins DC. Evaluating therapist adherence in motivational interviewing by comparing performance with standardized and real patients. J Consult Clin Psychol. 2014; 82(3):472-481.

2. Beidas RS, Cross W, Dorsey S. Show me, don’t tell me: behavioral rehearsal as a training and analogue fidelity tool. Cogn Behav Pract. 2014;21(1):1-11.

3. Walker DD, Stephens RS, Roffman R, Towe S, DeMarce J, Lozano B, Berg B. Randomized controlled trial of motivational enhancement therapy with nontreatment-seeking adolescent cannabis users: a further test of the teen marijuana check-up. Psych Addict Behav. 2011;25(3):474-484.

Источник: []

Notice: Undefined variable: z_bot in /sites/ on line 99

Notice: Undefined variable: z_empty in /sites/ on line 99


Leave a Reply

Your email address will not be published. Required fields are marked *