Line plot

Line plot displays n-grams as a series of data points connected by straight line segments.

The graph display unigrams (single words) and bigrams over a monthly or yearly period. It works best with homogenous datasets with relatively few periods (max T=10).

Coding example:

Use case: Essential topics in newspaper headlines

Data: Million News Headlines dataset, source: Australian Broadcasting Corporation, data licence: CC0 1.0: Public Domain.

Coding:

1import pandas as pd
2from arabica import cappuccino

1 data = pd.read_csv('abcnews_data.csv', encoding='utf8')

The data looks liks this:

headline

date

aba decides against community broadcasting licence

2003-2-19

act fire witnesses must be aware of defamation

2003-2-19

It procceeds in this way:

  • additional stop words cleaning, if skip is not None

  • lowercasing: reviews are made lowercase so that capital letters don’t affect n-gram calculations (e.g., “Tree” is not treated differently from “tree”), if lower_case = True

  • punctuation cleaning - performs automatically

  • stop words removal, if stopwords is not None

  • digits removal, , if numbers = True

  • n-gram frequencies for each headline are calculated, aggregated by monthly frequency, and displayed in a line plot.

 1cappuccino(text = data['headline'],
 2           time = data['date'],
 3           date_format = 'us',              # Uses US-style date format to parse dates
 4           plot = 'line',
 5           ngram = 1,                        # N-gram size, 1 = unigram, 2 = bigram
 6           time_freq = 'M',                  # Aggregation period, 'M' = monthly, 'Y' = yearly
 7           max_words = 6,                    # Displays 6 most frequent unigrams (words) for each period
 8           stopwords = ['english'],          # Remove English stopwords
 9           skip = ['covid','donald trump'],  # Remove additional stop words
10           numbers = True,                   # Remove numbers
11           lower_case = True)                # Lowercase text

Here is the output:

alternate text

Download the jupyter notebook with the code and the data here.