Welcome! Now that you have an understanding about twitter and how it can help us generate insights, let us jump right in and create a Twitter archive. In this module we will be discussing:
1. Framing a research question
2. Create an archive using TAGS
If you haven’t had a chance to download TAGS yet, follow this step-by- step guide.
1. Framing a research question
Before creating your archive it is important to select an appropriate research question. The research can be exploratory and need not have a specific hypothesis. For example, you may be interested in exploring reasons why women decided to leave or stay in a difficult relationship (Cravens et al, 2015) or understanding experiences of mental health service users (Shepherd et. al, 2015)
After framing your research question, narrow down a hashtag that is related to your research question. Alternately you could also use a case study approach and select an individual twitter account to retrieve tweets. Some important considerations while making this decision:
a. Select hashtags that cover larger topics and generate lots of activity, rather than more niche hashtags.
b. Since you can only collect data from last 6-9 days, you want to check the hashtag you are selecting that is in use NOW.
c. How a hashtag is used may change over time. For example during the George Floyd protests in June 2020 #BLM was being used to discuss racial violence. In May 2022, #BLM is associated with a much wider range of causes including promotion of black business and entrepreneurs.
2. Create an Archive using TAGS
Open your TAGS Google sheet. Select the first tab “Readme/Settings” on the sheet. Under the section “Instructions”, type in the hashtag or search string you wish to query for in Item 2. Enter term -> #mentalhealth. You could create an archive for tweets where two hashtags are used together. Type in both hashtags separated by the phrase “AND” . For eg. #mentalhealth AND #safespace. You could search for tweets related to a hashtag posted by a single user. For eg you can run a query for “#BLM AND from:BarackObama”.
TAGS also allows you to set additional criteria for guiding the archiving process for the app. Under Advanced Settings -> Follower count filter you can enter the minimum number of twitter account must have for inclusion in the archive. This can help in potentially removing tweets from fake accounts in our archive.
You can also specify the total number of tweets you want TAGS to gather for your archive Advanced Settings -> Number of tweets
You are now set.
Click on the “TAGS” menu on the top right corner and select “Run Now!” TAGS -> Run now! You will see a small message window pop-up “Running Script” on the page. Allow the pop -up message to disappear. This may take a minute or two. Your archive is now ready!
Head to the second tab “Archive” on your Google sheet (you will find this at the bottom center of the page) to see your tweets.
You will notice that the column headers of your archive tab have headings such as “id”, “from_user”, “text”.
Highlight and select all tweets (Ctrl +A -> Ctrl +C) and paste them on a new google sheet (Ctrl +V). Save this new sheet and use it for analyzing and coding your tweets.
Additional functions
IMP: If you want to run a new search using a different hashtag, head to the TAGS menu option. From the dropdown menu, select “Wipe Archive Sheet”, and delete all previous tweets. Save a copy using the method described above if you want to keep a copy of tweets from your previous query.
Rather than gathering all your tweets at one point of time, you can set up TAGS to run queries at set time intervals. Head over to TAGS -> “Update archive every hour” to keep collecting tweets automatically. The TAGS tool will continue issuing hourly calls to the Twitter REST API without your active intervention. To stop, head over to TAGS > Stop updating archive every hour.
Video Tutorial on using TAGS
I wish to than Francesca Giannetti for creating an excellent Github resource on Collecting Twitter data for Research. This project is inspired by a workshop she facilitated as part of the NYC Digital Humanities Week in 2021.
References
Cravens, J. D., Whiting, J. B., & Aamar, R. O. (2015). Why I stayed/left: An analysis of voices of intimate partner violence on social media. Contemporary Family Therapy, 37(4), 372-385.
Giannetti , F. (2022). Collecting Twitter data for Research. https://github.com/giannetti/collecting-twitter-data/blob/master/collecting-twitter-data.pdf
Shepherd, A., Sanders, C., Doyle, M., & Shaw, J. (2015). Using social media for support and feedback by mental health service users: thematic analysis of a twitter conversation. BMC psychiatry, 15(1), 1-9.
