Isolate String By Json Attribute

I'm building a twitter scraper that writes the search term to column 1 of a CSV and tweet search results to column 2. I've built the basic API function: def search_twitter(query, t

Solution 1:

You need something like as follows:

import csv

terms = ["example", "search terms", "go", "here"]

withopen("urls.csv", "w", newline="", encoding="utf-8") as f_out:
    writer = csv.writer(f_out)
    writer.writerow(["Search Term", "Tweet"])
    
    for term in terms:
        json_response = search_twitter(query=term, tweet_fields=tweet_fields, bearer_token=BEARER_TOKEN)
    
        for entry in json_response['data']:
            tweet = entry['text']
            tweet_lower = tweet.lower()
        
            if term in tweet_lower:
                writer.writerow([term, tweet])

This reads the list from the data item in the JSON. It then extracts the text item holding the tweet and compares it with each of your terms (lower() is used to make it case insensitive).

You would also probably need to encode the file with utf-8.

File location

The output file urls.csv should be located in your current working directory. This can be shown using:

import osprint(os.getcwd())

If you want, you could force the output to be in the same folder as the script using:

os.chdir(os.path.dirname(os.path.abspath(__file__)))