{"id":111698,"date":"2018-08-13T16:00:00","date_gmt":"2018-08-13T21:00:00","guid":{"rendered":"https:\/\/alpha.pewresearch.org\/pewresearch-org\/decoded\/\/\/an-intro-to-topic-models-for-text-analysis\/"},"modified":"2024-04-14T04:10:46","modified_gmt":"2024-04-14T09:10:46","slug":"an-intro-to-topic-models-for-text-analysis","status":"publish","type":"decoded","link":"https:\/\/alpha.pewresearch.org\/pewresearch-org\/decoded\/2018\/08\/13\/an-intro-to-topic-models-for-text-analysis\/","title":{"rendered":"An intro to topic models for text analysis"},"content":{"rendered":"\n
\"\"<\/a>
(Related posts: <\/em>Making sense of topic models<\/em><\/a>, <\/em>Overcoming the limitations of topic models with a semi-supervised approach<\/em><\/a>, Interpreting and validating topic models<\/em><\/a>, How keyword oversampling can help with text analysis<\/em><\/a> and <\/em>Are topic models reliable or useful?<\/em><\/a>)<\/em><\/figcaption><\/figure>\n\n\n\n

If you\u2019ve ever had to analyze a set of documents \u2014 such as social media posts, open-ended survey responses, focus group transcripts, software logs, etc. \u2014 you may have wondered about natural language processing (NLP). NLP is an umbrella term that encompasses a wide variety of algorithmic methods that can be used to automatically analyze large volumes of text.<\/p>\n\n\n\n

Some of these methods are \u201csupervised,\u201d which means that they require researchers to first classify a sample set of documents manually. Algorithms can then use these manual classifications to learn word associations well enough to apply the same system to a larger collection of documents. Supervised learning algorithms can save researchers a lot of time, but they still involve extensive and careful reading of documents and detailed categorization.<\/p>\n\n\n\n

Other NLP methods are \u201cunsupervised,\u201d which means they don\u2019t require training data classified by people. Instead, these algorithms look at how words are used in the documents you\u2019re examining. They pick up on patterns and provide an estimate of what the documents convey with little direct supervision from the researcher. One of the most popular forms of unsupervised learning for text analysis is topic models, which I\u2019ll be addressing here and in future posts.<\/p>\n\n\n\n

A topic model is a type of algorithm that scans a set of documents (known in the NLP field as a corpus), examines how words and phrases co-occur in them, and automatically \u201clearns\u201d groups or clusters of words that best characterize those documents. These sets of words often appear to represent a coherent theme or topic.<\/p>\n\n\n\n

There are a variety of commonly used topic modeling algorithms \u2014 including non-negative matrix factorization<\/a>, Latent Dirichlet Allocation<\/a> (LDA), and Structural Topic Models<\/a>. Ongoing research has produced other variants as well. At Pew Research Center, we\u2019ve been testing some of these algorithms and exploring how topic models might be able to help us analyze large collections of text, such as open-ended survey responses and social media posts. Here, I\u2019ll be exploring a dataset consisting of open-ended responses to a survey about what makes life feel fulfilling. All we need to train the model is a spreadsheet with a single column of text, one response (i.e. document) per row.<\/p>\n\n\n\n

\"\"<\/a><\/figure>\n\n\n\n

How topic models work<\/strong><\/h4>\n\n\n\n

In one way or another, every topic modeling algorithm starts with the assumption that your documents consist of a fixed number of topics. The model then assesses the underlying structure of the words within your data and attempts to find the groups of words that best \u201cfit\u201d your corpus based on that constraint. At the end, you\u2019re left with two output tables: the term-topic matrix, which breaks topics down in terms of their word components, and the document-topic matrix, which describes documents in terms of their topics.  Depending on the specific algorithm that you use, a word may be assigned to multiple topics in varying proportions, or assigned to a single topic exclusively.<\/p>\n\n\n\n

Let\u2019s consider the below example of a term-topic matrix, which is based on topics from a model trained on our open-ended responses. (The topics are real but for simplicity the numbers and documents here are illustrative and not based on actual data.) The first column contains an arbitrary identifier for each topic so we can refer to each topic by a name, followed by a column for every possible word that each topic could contain (known as our \u201cvocabulary\u201d). The values in the cells indicate how much each word \u201cbelongs\u201d to each topic. Their exact meaning will depend on the specific algorithm used, but usually the most common value in this table will be zero (or close to it), since only a fraction of our vocabulary will be at all relevant to any particular topic. Since these tables can become enormous, the example below shows just three topics and a few of their top words.<\/p>\n\n\n\n

\"\"<\/a><\/figure>\n\n\n\n

Since this raw output can be difficult to read and interpret, researchers often sort through the words for each topic and pick out the top words based on some measure of importance. For models that assign words to topics proportionally, we can look at the words that have the highest weights for each topic \u2014 that is, those that make up the greatest share of the topic. We can also use other metrics, like mutual information<\/a>, to compare the words in each topic against all of the other topics, and pick out the words that are most distinctive. Using one of these sorting methods, we can select the five or 10 most important words for each topic, making it easier for us to view and interpret them:<\/p>\n\n\n\n

\"\"<\/a><\/figure>\n\n\n\n

If our goal is simply to get a better sense of what\u2019s in our documents, then we may have already accomplished our mission. In cases where we have a large collection of text but don\u2019t really know the nature of its contents, topic models can help us get a glimpse inside and identify the main themes in our corpus. Topic 4 shows us that some of our documents have something to do with the pursuit of hobbies and passions for fun. Topic 5 seems to have something to do with charity and giving back to society, and Topic 88 touches on a more general notion of spending time pursuing and contributing to more serious things, like work. But you may also have noticed some quirks or irregularities in these topics. While each group of words seems to be capturing a distinct theme, all three are somewhat difficult to interpret precisely, and some words look a little out of place.<\/p>\n\n\n\n

Using topic models to classify documents<\/strong><\/h4>\n\n\n\n

Despite those quirks, we can still use this topic model for a rough categorization of individual documents, which can be useful if we want to make comparisons between documents and analyze how different themes are distributed. To do so, we need to examine the other side of the model\u2019s output, the document\u2013topic matrix.<\/p>\n\n\n\n

Most topic models break down documents in terms of topic proportions \u2014 for example, a model might say that a particular document consists 70% of one topic and 30% of another \u2014 but other algorithms may simply classify whether or not a document contains each topic. When using a model that allocates topics to documents as proportions, researchers can analyze the topics as either continuous variables, or as discrete classifications by setting a cutoff threshold to decide whether a document contains a topic or not. Here\u2019s an example of what the corresponding weights for our three example documents might look like, for the three different topics above:<\/p>\n\n\n\n

\"\"<\/a><\/figure>\n\n\n\n

By setting a 50% cutoff threshold, we can also turn these into categorical variables, like those you might collect in a survey:<\/p>\n\n\n\n

\"\"<\/a><\/figure>\n\n\n\n

When you first see topic model output, it can be inspiring. Having the ability to\u00a0automatically<\/em>\u00a0identify and measure the main themes in a collection of documents opens the door to all kinds of exciting research questions, and you may find yourself eager to get started. However, it turns out that running the topic model is the easy part \u2014 and the first of several steps. Before we can use our model as a measurement tool, we also have to figure out what it\u2019s measuring \u2014 an undertaking that can be surprisingly difficult. In\u00a0my next post<\/a>, I\u2019ll explain why.<\/p>\n","protected":false},"excerpt":{"rendered":"

Topic models can scan documents, examine words and phrases within them, and \u201clearn\u201d groups of words that characterize those documents.<\/p>\n","protected":false},"author":655,"featured_media":0,"comment_status":"closed","ping_status":"closed","template":"","meta":{"sub_headline":"","sub_title":"","_prc_public_revisions":[],"_ppp_expiration_hours":0,"_ppp_enabled":false,"ai_generated_summary":"","relatedPosts":[],"datacite_doi":"","datacite_doi_citation":"","_prc_seo_qr_attachment_id":0,"spoken_article_player_enabled":true,"displayBylines":true,"footnotes":"","prc_watchers":[],"_prc_fork_parent":0,"_prc_fork_status":"","_prc_active_fork":0},"categories":[353],"bylines":[927],"collection":[],"_post_visibility":[],"decoded-category":[530],"formats":[],"_fund_pool":[],"languages":[],"regions-countries":[],"research-teams":[524],"workflow-status":[],"class_list":["post-111698","decoded","type-decoded","status-publish","hentry","category-data-science","bylines-patrick-van-kessel","decoded-category-data-science","research-teams-decoded"],"label":"Decoded","post_parent":0,"word_count":1262,"canonical_url":"https:\/\/alpha.pewresearch.org\/pewresearch-org\/decoded\/2018\/08\/13\/an-intro-to-topic-models-for-text-analysis\/","art_direction":{"A1":{"id":126061,"rawUrl":"https:\/\/alpha.pewresearch.org\/pewresearch-org\/wp-content\/uploads\/sites\/20\/2022\/08\/08.13.2018_featured.png","url":"https:\/\/alpha.pewresearch.org\/pewresearch-org\/wp-content\/uploads\/sites\/20\/2022\/08\/08.13.2018_featured.png?w=564&h=317&crop=1","width":564,"height":317,"caption":"","chartArt":false},"A2":{"id":126061,"rawUrl":"https:\/\/alpha.pewresearch.org\/pewresearch-org\/wp-content\/uploads\/sites\/20\/2022\/08\/08.13.2018_featured.png","url":"https:\/\/alpha.pewresearch.org\/pewresearch-org\/wp-content\/uploads\/sites\/20\/2022\/08\/08.13.2018_featured.png?w=268&h=151&crop=1","width":268,"height":151,"caption":"","chartArt":false},"A3":{"id":126061,"rawUrl":"https:\/\/alpha.pewresearch.org\/pewresearch-org\/wp-content\/uploads\/sites\/20\/2022\/08\/08.13.2018_featured.png","url":"https:\/\/alpha.pewresearch.org\/pewresearch-org\/wp-content\/uploads\/sites\/20\/2022\/08\/08.13.2018_featured.png?w=194&h=110&crop=1","width":194,"height":110,"caption":"","chartArt":false},"A4":{"id":126061,"rawUrl":"https:\/\/alpha.pewresearch.org\/pewresearch-org\/wp-content\/uploads\/sites\/20\/2022\/08\/08.13.2018_featured.png","url":"https:\/\/alpha.pewresearch.org\/pewresearch-org\/wp-content\/uploads\/sites\/20\/2022\/08\/08.13.2018_featured.png?w=268&h=151&crop=1","width":268,"height":151,"caption":"","chartArt":false},"XL":{"id":126061,"rawUrl":"https:\/\/alpha.pewresearch.org\/pewresearch-org\/wp-content\/uploads\/sites\/20\/2022\/08\/08.13.2018_featured.png","url":"https:\/\/alpha.pewresearch.org\/pewresearch-org\/wp-content\/uploads\/sites\/20\/2022\/08\/08.13.2018_featured.png?w=720&h=405&crop=1","width":720,"height":405,"caption":"","chartArt":false},"social":{"id":126061,"rawUrl":"https:\/\/alpha.pewresearch.org\/pewresearch-org\/wp-content\/uploads\/sites\/20\/2022\/08\/08.13.2018_featured.png","url":"https:\/\/alpha.pewresearch.org\/pewresearch-org\/wp-content\/uploads\/sites\/20\/2022\/08\/08.13.2018_featured.png?w=1200&h=628&crop=1","width":1200,"height":628,"caption":"","chartArt":false}},"_embeds":[],"watchers":[],"table_of_contents":[],"datacite_doi":"","prc_seo_data":{"title":"An intro to topic models for text analysis","description":"Topic models can scan documents, examine words and phrases within them, and \u201clearn\u201d groups of words that characterize those documents.","og_title":"An intro to topic models for text analysis","og_description":"Topic models can scan documents, examine words and phrases within them, and \u201clearn\u201d groups of words that characterize those documents.","schema_type":"Article","noindex":false,"canonical_url":"","primary_terms":{"category":1},"custom_schema":[],"og_image":126061,"indexnow_submitted_at":null,"gsc_index_status":null},"prepublish_checks":{},"jetpack_sharing_enabled":true,"relatedPostsOrdered":[],"bylinesOrdered":[{"key":"_e58b3a3u8","termId":927}],"acknowledgementsOrdered":[],"_links":{"self":[{"href":"https:\/\/alpha.pewresearch.org\/pewresearch-org\/wp-json\/wp\/v2\/decoded\/111698","targetHints":{"allow":["GET"]}}],"collection":[{"href":"https:\/\/alpha.pewresearch.org\/pewresearch-org\/wp-json\/wp\/v2\/decoded"}],"about":[{"href":"https:\/\/alpha.pewresearch.org\/pewresearch-org\/wp-json\/wp\/v2\/types\/decoded"}],"author":[{"embeddable":true,"href":"https:\/\/alpha.pewresearch.org\/pewresearch-org\/wp-json\/wp\/v2\/users\/655"}],"replies":[{"embeddable":true,"href":"https:\/\/alpha.pewresearch.org\/pewresearch-org\/wp-json\/wp\/v2\/comments?post=111698"}],"version-history":[{"count":2,"href":"https:\/\/alpha.pewresearch.org\/pewresearch-org\/wp-json\/wp\/v2\/decoded\/111698\/revisions"}],"predecessor-version":[{"id":138558,"href":"https:\/\/alpha.pewresearch.org\/pewresearch-org\/wp-json\/wp\/v2\/decoded\/111698\/revisions\/138558"}],"wp:attachment":[{"href":"https:\/\/alpha.pewresearch.org\/pewresearch-org\/wp-json\/wp\/v2\/media?parent=111698"}],"wp:term":[{"taxonomy":"category","embeddable":true,"href":"https:\/\/alpha.pewresearch.org\/pewresearch-org\/wp-json\/wp\/v2\/categories?post=111698"},{"taxonomy":"bylines","embeddable":true,"href":"https:\/\/alpha.pewresearch.org\/pewresearch-org\/wp-json\/wp\/v2\/bylines?post=111698"},{"taxonomy":"collection","embeddable":true,"href":"https:\/\/alpha.pewresearch.org\/pewresearch-org\/wp-json\/wp\/v2\/collection?post=111698"},{"taxonomy":"_post_visibility","embeddable":true,"href":"https:\/\/alpha.pewresearch.org\/pewresearch-org\/wp-json\/wp\/v2\/_post_visibility?post=111698"},{"taxonomy":"decoded-category","embeddable":true,"href":"https:\/\/alpha.pewresearch.org\/pewresearch-org\/wp-json\/wp\/v2\/decoded-category?post=111698"},{"taxonomy":"formats","embeddable":true,"href":"https:\/\/alpha.pewresearch.org\/pewresearch-org\/wp-json\/wp\/v2\/formats?post=111698"},{"taxonomy":"_fund_pool","embeddable":true,"href":"https:\/\/alpha.pewresearch.org\/pewresearch-org\/wp-json\/wp\/v2\/_fund_pool?post=111698"},{"taxonomy":"languages","embeddable":true,"href":"https:\/\/alpha.pewresearch.org\/pewresearch-org\/wp-json\/wp\/v2\/languages?post=111698"},{"taxonomy":"regions-countries","embeddable":true,"href":"https:\/\/alpha.pewresearch.org\/pewresearch-org\/wp-json\/wp\/v2\/regions-countries?post=111698"},{"taxonomy":"research-teams","embeddable":true,"href":"https:\/\/alpha.pewresearch.org\/pewresearch-org\/wp-json\/wp\/v2\/research-teams?post=111698"},{"taxonomy":"workflow-status","embeddable":true,"href":"https:\/\/alpha.pewresearch.org\/pewresearch-org\/wp-json\/wp\/v2\/workflow-status?post=111698"}],"curies":[{"name":"wp","href":"https:\/\/api.w.org\/{rel}","templated":true}]}}