Web Scraping Economics Journals info from Elsevier
Date:
Description:
This project initially served as a research data source for my working paper titled “Attraction is All You Need: Can AI-Predicted Eye-Catching Titles Spell Academic Triumph?” It primarily utilizes Python’s Playwright library to scrape essential details from economics journals on Elsevier. The information scraped includes paper titles, abstracts, authors, author affiliations, citation counts, publication dates, and the publication process. The specific code can be found in a GitHub repository, with the contents of each file described as follows:
- Web_scraping.py: Contains the main script for scraping data using Playwright.
- data/basic_information/: Directory containing the initial data files used by the scraper to identify which papers need to be fetched.
- data/dataset/: Contains the results of the scraping process, with detailed information for each paper that has been successfully scraped.
Repositries:
Elsevier Economics Papers Scraper
Related with:
“Attraction is All You Need: Can AI-Predicted Eye-Catching Titles Spell Academic Triumph?”