Posted by Nodus Labs | September 10, 2018
By Dmitry Paranyushkin, Nodus Labs, Published April 2018
Download the PDF Version
Diagrams and visual maps used to represent information as a network of interconnected nodes are widely used to aid one’s understanding of data and produce novel interpretations of the already known facts. However, little is known about the effect of graph representation on the processes of ideation. Specifically, what is the difference between the text, which is represented as a sequence of words, as a narrative, and the text, which is represented as a graph? We studied the effect of text network graph representation on the processes of ideation and found that direct live visual feedback increases the subjective sense of consistency in one’s thoughts and increases connectivity between the ideas expressed.
Keywords: TNA, interfaces, cognition, text network analysis, graphs, education, learning, comprehension, visualisation, connectivity
1.1 Diagrams and Interfaces as Heuristic Devices
Diagrams and visual maps for representing information have been shown to be effective tools for text analysis, better text memorisation and learning. Several studies have shown that diagrams can provide a better idea about the narrative structure within the text (Bruce et al 1978, Lehnert 1981, Dyer 1983). Use of schemas, particularly knowledge graphs, have been shown to aid better understanding of textual data and tackle the dynamic nature of knowledge (Popping 2003). Narrative schemas can be used not only to better understand text, but to also describe similarities between different stories (Loewe 2010). Network-like structures used to represent and better understand ideas are commonly referred to as mental models or mental maps (Minsky 1975; Johnson-Laird 1986; Sowa 1992). The nodes are the concepts and the connections the relations between them. Such mental models can be extracted from interviews to produce the cognitive maps, which can then be used for group sentiment analysis (Carley & Palmquist 1992; Carley 1997; Jonassen & Cho Y.H. 2008). The concept of lexical priming emphasises the notion of collocation (co-occurrence): that is, a tendency of words to be found in close environment to one another (Hoey 2005, Pace-Sigge 2013). It has also been shown that semantic priming (Neely 1977, Foss 1982) — a process by which words tend to be recognised faster when used with the words that have a close semantic proximity to them – plays an important role in memory processes (Heyman et al 2015). Therefore, word co-occurrence graphs do not only represent the semantic proximity of terms, but also can themselves be used to facilitate the process of semantic priming (Evert & Lapesa 2013, Bullinaria & Levy 2007).
Further research (Doyle et al 2007) has shown how a similar approach can be used to measure change in mental models of complex dynamic systems. Ryan (2007) proposed to think of a diagram as an heuristic device, which can represent a narrative in a spatial, temporal and mental plane, thus allowing for multiple readings and novel interpretation of textual data. Many different types of text graph visualizations (Lima 2011; Paley 2002; Carley 1997) have been designed to get additional insight into texts or their underlying mental maps. Concept maps have been shown to improve students’ performance (Hwang et al 2014) and facilitate group learning (Wang et al 2017).
Studies cited above show that diagrams and visualisation of the relations within data can be very effective for better understanding of complex domains of knowledge and learning.
However, there is a lack of research on the effect the diagrams and visualisations have on this data, information, knowledge, and the cognitive process itself. While it is well known that all those tools can help understand a certain domain of knowledge, little is known about the nature of this understanding, and, more importantly, how using those tools can actually transform and advance the knowledge itself.
It is our hypothesis that word co-occurrence graphs may have a positive effect on comprehension and memorisation due to the semantic priming effects described earlier. In an experimental setup we will analyse the effects that network graph visualisation of word co-occurrences has on the cognitive process.
1.2 The Effects of Interfaces on Cognition and Knowledge
The effects of interfaces used for sharing, processing, and retaining information are well studied. The most publicised effect is the “filter bubble”, where recommendation algorithms based on preferential proximity show only the information that has received considerable positive attention from the user’s immediate social circles (Pariser 2011). This, in effect, produces homogeneous informational, cultural, and ideological domains, hiding the different points of view and, thus, stifling creative and innovative processes.
Social network interfaces, such as the Facebook’s newsfeed, are known to have an effect on human behavior. It has been demonstrated that over the period from 2009 to 2014 users preferred shorter and simpler activities on Facebook as the social network’s newsfeed was specifically redesigned to encourage this behaviour (Paul et al 2015). Studies suggest that social network interface design can in fact be used to control user’s emotional state and behavior (Kramer et al 2014). In this context it may be important to explore how digital interfaces can be designed to stimulate and enhance cognitive capacities, imagination and learning process.
Various computer interfaces such as team chats or digital whiteboards are often used for ideation processes and promote collective mindfulness in organisations (Curtis et al 2017). They are known to function better when they reduce the cognitive load (Ovlatt, 2004) although even a higher cognitive load may lead to better performance among students (Hwang et al 2014). Studies have also shown that task-specific interfaces reduce the load on the doctors in critical care units (Ahmed 2011).
In order to cope with informational overload several interface solutions have been introduced over the past years, which are specifically designed to improve comprehension and learning. Tagclouds — visual representation of the most frequently used words in a text — have been shown to improve searching and recognition (Rivadeneira et al 2017; Singh et al 2017).
Written text in itself can also be seen as an interface and has certain effects on cognition described by Van Den Broek (1995) using the “landscape reading model” (LRM). LRM is based on the premise that, during reading, the ideas and concepts associated with the text fluctuate in their activation. Therefore the level of reader’s perception of text will depend on referential and causal coherence invoked by the combinations of concepts within text: e.g. a more thorough knowledge of the topic’s background will ease activation of certain topics in relation to one another and, thus, will aid comprehension.
In this study we look at a specific type of interfaces for information processing, which use visual network / graph representations of interconnected data. This is especially relevant today as the increasing number of online corporations are using the graph model for back-end information storage, however information representation is still done in categorised lists and popularity-ranked newsfeed.
1.3 The Effects of Network and Graph Representation on Cognition
The objective of this study is to evaluate the effects of graph representation on information processing and cognition for end users. Specifically, we want to compare standard chronological list representation (e.g. standard text) versus graph representation of the same data and observe the effects these two representations have on the ideation process. We also want to see how users interact with the graphs representing their knowledge and how making the existing structural gaps (Burt 2009) visible in the graphs may affect ideation and creativity.
A study by Noy et al (2012) have shown that so-called “creative leaps” occur when there is a sudden shift from one plane of possibilities to another. We would like to apply this metaphor to the way people interact with graphs and estimate whether bridging the gap between two disjointed clusters within a network representing a certain domain of knowledge would lead to creating new ideas. We believe that if our hypothesis proves to be valid, several techniques for information notation, writing, and creative ideation could be developed based on graph interfaces.
2. Experimental Setup
2.1 The Model
In order to estimate the effects of graph interfaces on the processes of cognition and ideation as well as its influence on one’s domain of knowledge, we developed a special software tool InfraNodus (online version: www.infranodus.com, source code: http://github.com/noduslabs/infranodus) used in the experimental setup.
This tool can record a series of statements about a certain domain of knowledge. The words in each statement are normalised (Paranyushkin 2011b) and then visualised on the graph as the nodes. When the words occur within the same statement, they are connected to one another with edges in the graph (Paranyushkin 2011a) (see Fig 2).
The nodes (words) that have a higher betweenness centrality measure are shown bigger on the graph, indicating the main topics in the text.
Force-atlas layout is applied to the resulting graph (Jacomy 2009), which pushes the most connected nodes (words) apart from each other, while the nodes that are connected to those most connected nodes are pulled towards them. This layout creates a very clear visual representation of the communities contained within the graph and represents the clusters of words that appear more frequently next to each other within the series of statements.
Finally, community detection mechanism (Blondel et al 2007) is applied to group the nodes (words) into communities based on how dense the links are between them.
As a result we get a visualisation of the main topics inside the text and how they are related to one another (Paranyushkin 2011). We can also see the structural gaps in the graph, which may potentially be used to generate new ideas (Noy et al 2012; Burt 2009).
Visually this approach is similar to tagclouds, where the most frequently used words are emphasised visually. The main difference here is that betweenness centrality takes into account not only the word’s frequency, but also how well it’s embedded into the whole text. The more different distinct parts of a network the node connects, the higher its betweenness centrality is. So this measure also indicates how relevant the node is for overall discourse formation (Corman 2002). Furthermore, text network graph shows how the most influential words are related to one another, something that is absent in tag clouds. This provides users with the context for every element of the graph, making it potentially a much more useful heuristic device.
For our experiment we recruited 17 participants through social networks and the word-of-mouth (8 females and 9 males, 20 to 58 mean age, diverse social, cultural, geographical, professional and income backgrounds). In order to not influence the results, we invited them for a 2-hour interview about their interests without telling them about the real intent of the experiment.
The experiment was conducted in a private, neutral setting, where the participant was left one on one with the person conducting the experiment. The participants were asked to speak about the subject of their current interest or research in short statements (stage 0). Each statement was recorded using InfraNodus tool as a text and also as a visualised graph following the procedure described earlier.
At the first stage (the participants did not see the graph interface or the computer screen and were just talking to the interviewer (operator) about a subject of their current interest or research. The operator was recording their statements chronologically.
After 20 minutes (or shorter, if the participant decides that the topic is fully explored) – stage 1a – the participants were asked to look at the text they wrote (without seeing the graph) and asked if they want to reformulate something or add something to the text (stage 1b).
After modifications were made (or not – depending on the wish of the participants), we would then show them the graph, explain the basic algorithm behind its formation (the nodes are the words, the edges are their co-occurrences), and ask if they had something to add or amend to their text represented as the graph. This stage would last 10 minutes (stage 1c). This stage allowed us to observe the effect graph representation has on one’s overview of the conversation’s subject.
At the second stage (stage 2) of experiment (lasting maximum 30 minutes or until the participant decides that the topic is fully explored), we asked the participants to speak about another subject, while showing them both the statements they were saying and the graph visualisation produced from those statements simultaneously. We asked them to consult the graph during the second interaction in case they find it helpful. The results of this stage of the experiment are shown in Table 1. A sample graph from the experiment is shown on Figure 1.
At the third stage of the experiment (stage 3), we showed participants the both graphs: one from the first interview and one from the second. We would then ask them to tell us if they had anything to add or change in relation to those two graphs. The next question would purposefully relate two-three most connected concepts in the first graph (stage 1) to two-three most connected concepts in the second graph (stage 2) – functioning as a proposition to make connections between things.
The obtained graphs were then exported into gexf format (a graph-centric type of xml format) using InfraNodus in order to be later analysed using Gephi network analysis software. The basic metrics we obtained was the number of nodes and edges, and various measures of graphs’ interconnectedness and structure, in order to have quantitative data for our study.
At the final stage of experiment we asked the participants about their experience using the graph when formulating their thoughts in order to get qualitative data. The experimenters would also record their experience.
2.3.1 Filling the Gaps. All the 17 participants reported to have understood the task well and were talking about the field of their current interest following the procedure proposed as described in stage 1a (using short statements). When shown the text (without showing the graph) and asked if they had anything to amend or add (stage 1b of the experiment), only 2 out of 17 participants wanted to add a statement to clarify the subject and 2 participants asked to edit the existing statements to make them more precise.
However, when the graph was shown (stage 1c), 9 out of 17 participants asked to add a statement and 5 other participants asked to edit the statements to clarify their ideas. Both newly added and edited statements would either fill in the gaps between the clusters of nodes shown on the graph or emphasise (increase the number of connections, and, thus, the size) of the node (keyword) that the participant considered important in the subject they were talking about.
2.3.2 Making Connections. Further quantitative analysis of the graph structure before the participants were shown the graph (stage 1b) and after the graph-based amendments were made (stage 1c) showed that its density and the average number of connections was increased, indicating increased connectivity.
During the second stage of the experiment the participants were talking about another subject of their interest while seeing the graph. We observed similar behaviour as shown in transition between the stage 1b to stage 1c (shown on Figure 3). Seeing the graph and connections between the keywords used in their statements during the interview, all the 17 participants were focusing on
1) making the graph more connected through adding new statements that linked the concepts that were not previously connected – often those shown bigger on the graph (i.e. higher number of connections).
2) emphasising the concepts that seemed subjectively important to the participants (in their subject), but not shown big enough on the graph (because of the few connections), by making the new statements that would incorporate those concepts and connect them to the ones already presented on the graph.
2.3.3 Coherence. The majority of participants (14 out of 17) seemed to be satisfied with the contents and appearance of the graph when its connectivity reached a level that was consistent for all the participants.
Detailed analysis of the graph’s metrics (Table 1) shows that the main graph parameters for nearly all the participants stay within 1 standard deviation from the mean, indicating that there is a certain correlation between the well-formed state of the subject and the basic text graph metrics. Interestingly, those include modularity value of around 0.42 in average, which, according to the modularity algorithm used (Blondel 2007) indicates that there is a distinct community structure within the graph. A value below 0.4 would indicate that the graph nodes tend to belong to one community (i.e. no nodes are more densely connected together than with the rest of the graph). Also, none of the graphs had several components, all the participants produced graphs that had only one major component.
2.3.4 Better Overview. Further conversations with the participants (stage 3) and qualitative assessments by operators revealed that nearly all of the participants (15 out of 17) would use that graph to compare their intention (and subjective understanding of what was important) to the more “objective” representation of the graph (according to the participants). They would then try to add things to either emphasise what they thought was important, but was not reflected in the graph (which means creating more connections for a concept or a cluster that appears underdeveloped on the graph – i.e. having not too many connections) or connect concepts and clusters together (by making new statements that would create links between distinct clusters on the graph.
Another observation common for all the participants was that seeing the graph did not really influence the strategies they used for building the narrative. However, it did provide them a visual summary of what has been said so far, making it easier (according to the participants) to have an overview of the whole text and all main topics inside. This may be helpful for ideation process as it has been shown that groups that are assigned different topics within the same context are more effective in producing ideas than specialised groups that focus on only one (Baruah & Paulus, 2011)
13 out of 17 participants reported that having the graph interface in front of them allowed them to formulate their thoughts more precisely.
10 out of 17 participants reported that the process of building the graph (stage 2) informed their understanding of the subject they were talking about much more than seeing the end result.
6 out of 17 participants reported that visual shapes produced in the graph (via the force-atlas layout algorithm) have in fact provided them additional visual inspiration for their understanding of the subject they were talking about.
Asking the questions that would purposefully target the structural gaps within the existing graphs and between the graphs (stage 3 of the experiment) would always produce a sort of creative leap (emergence of a new idea related to one or both of the subjects) for all the participants.
From the analysis above we can conclude several effects of live graph feedback (showing relations between the keywords based on their proximity and co-occurrence) on the processes of ideation and explanation, expressed through writing and talking.
- Having direct visual feedback through the graph interface encourages one to make more connections between disjointed ideas. (2.3.1 and 2.3.2)
- There is a correlation between the text graph metrics and how well-formed (subjectively) the text is to the user (2.3.3). From the initial analysis of the data obtained it can be stated that the topic is explored well (subjectively) when the resulting text graph has a prominent community structure, however, where those communities are well-connected (almost on the verge of merging into one graph). Also, all the participants generated a graph with only 1 main component, indicating that graph interfaces encourage increased connectivity between ideas (as stated in 1 above).
- Graph interface functions as a sort of “objective” (even if biased) representation of thought-flow, thus motivating the user to bridge the gap between their subjective understanding of a subject and its “objective” graph representation. (2.3.4)
- Graph interface helps people express themselves in a manner that’s more precise and consistent (using the words of the participants). (2.3.4)
- Graph interface is especially helpful in the process of writing, not after it has been done, especially because of the influence it has promoting making more connections between ideas while giving an overview of the already existing relations while thinking and writing. (2.3.1, 2.3.3, 2.3.4)
- Creative leaps and novel ideas seem to occur at the moments when the participants make a statement that bridges two distinct, but disconnected clusters (or connected nodes) in the graph. A question asked in a similar manner, addressing those structural gaps in the graph, will often have a similar effect. (2.3.4)
In this study we have shown that graph interfaces have a direct effect on the processes of ideation and cognition. Specifically, live feedback of the narrative as a graph encourages the narrator to connect disjointed ideas together. It also encourages one to build the narrative so that there are several interconnected clusters of meanings (or subtopics) present within the text. Narrators who had the graph representation of their text live as the text was being created also reported an increased sense of consistency and coherence in their thought process, easing the process of ideation for them, at least on the subjective level. Graph visualisation of text was also reportedly used as a sort of “objective” representation of narrative, indicating a high level of trust to such interfaces even if the inner mechanisms of their work are not fully known to the users. Finally, posing questions which would purposefully address the structural gaps within the graph would often lead to discoveries and novel ideas on the side of participants.
Our findings suggest that text graph representations could encourage and enhance the learning process, help formulate ideas and make texts more coherent. Also, some of our results suggest that graph interfaces could be used to help generate novel ideas when the users or interviews focus on structural gaps in the graphs, posing the questions aimed at bridging previously disconnected clusters and nodes (concepts). Finally, our results have also shown that graph interfaces have a visible and quantifiable effect on the processes of ideation and those effects should be studied more extensively with the proliferation of graph interfaces and tools in online platforms.
Further research could be made to study the correlations between the intention of the participants narrating a subject of their interest (e.g. explanation vs inspiration). It is our assumption that very low deviation in graph metrics between the different participants indicates that the intention to “explain” produces a certain structure of semantically well-formed graphs, while a different intention could produce very different graph structures (e.g. disjointed components, higher modularity, lower connectivity, etc.)
Also, it would be important to perform a study of graph’s effects on readers, not writers, in order to fully understand the effects that text network representation has on the processes of understanding and comprehension of textual data.
Ahmed, A; Herasevich, V; Pickering, B; (2011) The effect of two different electronic health record user interfaces on intensive care provider task load, errors of cognition, and performance. Critical Care Medicine, Volume 39, Issue 7, pp 1626-1634
Baruah, J; Paulus, P; (2011) Category assignment and relatedness in the group ideation process. Journal of Experimental Social Psychology, Volume 47, issue 6, pp 1070-1077
Blondel, V.; Guillaume, J-L; Lambiotte, R; Lefebvre, E; (2008). Fast Unfolding of Communities in Large Networks. In Journal of Statistical Mechanics: Theory and Experiment, Volume 2008
Bruce, D; Newman, D. (1978). Interacting Plans. In Cognitive Science, Volume 2, Issue 3, July-September: 195-233
Bullinaria, J, Levy, J (2007). Extracting semantic representations from word co-occurrence statistics: A computational study. Behavior Research Methods (2007) 39: 510. https://doi.org/10.3758/BF03193020
Burt, R (2009). Structural holes: The social structure of competition.
Carley, K and Palmquist, M (1992). Extracting, Representing, and Analyzing Mental Models. Institute for Software Research. Paper 40.
Carley, K. (1997). Extracting Team Mental Models through Textual Analysis. Journal of Organizational Behavior
Corman, S; Kuhn, T; McPhee, R; Dooley, K. (2002). Studying Complex Discursive Systems: Centering Resonance Analysis of Communication. Human Communication Research, Volume 28, Issue 2, pp 157-206
Curtis, Aaron M.; Dennis, Alan; and McNamara, Kelly O.. 2017. “From Monologue to Dialogue: Using Performative Objects to Promote Collective Mindfulness in Computer-Mediated Team Discussions,” MIS Quarterly, (41: 2) pp.559-581.
Doyle, J.; Radzicki, M; Trees, S; (2008). Measuring Change in Mental Models of Complex Dynamic Systems. Springer
Dyer, G. (1983). The Role of Affect in Narratives. In Cognitive Science 7: 211-242,
Foss, D (1982). “A discourse on semantic priming”. Cognitive Psychology. 14: 590–607.
Heyman, T., Van Rensbergen, B., Storms, G., Hutchison, K. A., & De Deyne, S. (2015). The influence of working memory load on semantic priming. Journal of Experimental Psychology: Learning, Memory, and Cognition, 41(3), 911-920.
Hwang G, Kuo F, Chen N, Ho H. (2014). Effects of an integrated concept mapping and web-based problem-solving approach on students’ learning achievements, perceptions and cognitive loads. Computers & Education, Volume 71, February 2014, Pages 77-86
Jacomy, M. (2009). Force-Atlas Graph Layout Algorithm. URL: http://gephi.org/2011/forceatlas2-the-new-version-of-our-home-brew-layout/
Jonassen D., Cho Y.H. (2008) Externalizing Mental Models with Mindtools. In: Ifenthaler D., Pirnay-Dummer P., Spector J.M. (eds) Understanding Models for Learning and Instruction. Springer, Boston, MA
Johnson-Laird, P.N. (1986). Mental models: towards a cognitive science of language, inference, and consciousness. Harward University Press, Issue 6 of Cognitive Science series
Kramer, A.; Guillory, J; Hancock, J; (2014). Experimental evidence of massive-scale emotional contagion through social networks. National Academy of Sciences, May 29, 2014.
Lapesa, G, Evert, S. (2013). Evaluating Neighbor Rank and Distance Measures
as Predictors of Semantic Priming. Proceedings of the Workshop on Cognitive Modeling and Computational Linguistics, pages 66–74, Sofia, Bulgaria, August 8, 2013.
Lehrnert, W. (1981). Plot Units and Narrative Summarization. In Cognitive Science, Volume 5, Issue 4: 293–331
Lima, M. (2011). Visual Complexity: Mapping Patterns of Information. Princeton Architectural Press.
Loewe, B. (2010). Comparing Formal Frameworks of Narrative Structure. In Association for the Advancement of Artificial Intelligence
Minsky, M (1974). A Framework for Representing Knowledge. In Artificial Intelligence, Memo no. 306
Neely, J. H. (1977). Semantic priming and retrieval from lexical memory: Roles of inhibitionless spreading activation and limited-capacity attention. Journal of Experimental Psychology: General, 106(3), 226-254.
Noy, L, Hart, Y, Andrew, N, Ramote, O., Mayo, A. Alon, U. (2012). A quantitative study of creative leaps. International Conference on Computational Creativity
Ovlatt, S. (2004). Human-centered design meets cognitive load theory: designing interfaces that help people think. MM ’06 Proceedings of the 14th ACM international conference on Multimedia, pp 871-880
Pace-Sigge, T.L. M. (2013). The concept of lexical priming in the context of language use. ICAME Journal
Paley, B.W. (2002). TextArc text visualization software. URL: www.textarc.org
Paranyushkin, D (2011). Identifying the pathways for meaning circulation using text network analysis.
Paranyushkin, D (2011). Visualization of text’s polysingularity using network analysis
Pariser, E (2011). The filter bubble: What the Internet is hiding from you.
Paul, T; Puscher, D; Strufe, T. (2015). The User Behavior in Facebook and its Development from 2009 until 2014
Popping, R. (2003). Knowledge Graphs and Network Text Analysis. In Social Science Information
Ryan, M-L. (2007). Diagramming Narrative. In Semiotica 165–1/4: 11–40.
Singh, J; Zerr, S; Siersdorfer, S; (2017). Structure-Aware Visualization of Text Corpora. Proceedings of the 2017 Conference on Conference Human Information Interaction and Retrieval, pp 107-116
Sowa, J. (1992). Semantic Networks. In Encyclopedia of Cognitive Science
Wang, M. Cheng, B. Chen, J. Mercer, N. Kirschner, P. (2017) The use of web-based collaborative concept mapping to support group learning and interaction in an online environment. The Internet and Higher Education, Volume 34, July 2017, Pages 28-40