In the ever-evolving landscape of web automation, finding innovative solutions to common challenges is key. Today, we'll explore a clever pagination solution for websites like TeePublic that don't feature the conventional "Next Page" button but instead rely on numerical navigation.

Setting the Stage: Imagine navigating a website without the familiar "Next" button and instead encountering a series of page numbers. Our journey begins with the realization that we can leverage Google Sheets to streamline this process.

Utilizing Google Sheets for Pagination: By compiling a list of URLs in a Google Sheet, we can seamlessly loop over the pages without the need to manually click through each number. With a simple click, the URL changes to reflect the desired page, allowing for efficient navigation up to page 100 and beyond.

Creating Automated Workflows: To streamline the process further, we delve into creating a new automation workflow. By employing web automation tools and utilizing suggested steps, we can scrape essential data such as names, prices, and URLs from the website.

Enhancing Data Extraction: Despite potential challenges like obscured text within tables, our automation remains robust. By customizing the scraping process to extract titles, URLs, and authors efficiently, we ensure comprehensive data retrieval.

Implementing List Triggers: The automation workflow is completed by integrating a list trigger with a Google Sheet. This integration allows for the seamless transition between URLs and enables batch processing for efficient data extraction.

Customizing Data Output: To conclude the process, we configure the output data to be exported to a designated Google Sheet. By defining specific headers and fields such as title, author, and URL, we can organize and store the scraped information systematically.

Expanding Possibilities: The versatility of this automation extends beyond data scraping, offering endless possibilities for further customization. Whether extracting images, additional details, or performing subsequent actions based on the obtained data, the automation can adapt to various use cases.

Conclusion: Innovative solutions like leveraging Google Sheets for pagination exemplify the power of automation in streamlining processes and enhancing efficiency. By harnessing the capabilities of web automation tools, we can revolutionize how we interact with and extract data from websites with non-traditional navigation interfaces.

Step1- Add the links to Google sheet

Step 2- Add the links up to 100 in Google sheet

Step 3- Click on New Automation

Step 4- Click on Web

Step 5- Click on Continue without cookies

Step 6- Enter the first URL in link box and click on Save go to URL

Step 7- Click on Scrape a list

Step 8- Select Name, link, Price, other information and add in respective column A, B, C, D Then Click on Confirm

Step 9- Click on I m done

Step 10- Select automation@taskmagic.com — Click on Send

Step 11- Click on Trigger and Click on List

Step 12- Check mark Use list from a google sheet — Click on Setup sheet connection

Step 13- Paste the Spread sheet — Select list —Click on Looks good

Step 14- Change Row and Loop number accordingly — Click on Continue

Step 15- Edit go to page— Change the URL — Click on Continue

Step 16- Create new sheet and Segregate with title

Step 17- Click on Send to Google Sheets —Paste the spreadsheet link—Select Output—Click on Looks good, This will import all the data to sheet

So with the website, Tee Public, because they don't have a next page button. Oh, I thought that was next for a second. Um, because they don't have a next page button and they use these numbers instead. What I realized is, is we can use a listing from a Google sheet to loop over URLs instead. So you'll see when I click page two, this changes this to page two.

And then if I change this to page four, that would take us to page four. Right. So what we can do is, is we can add all of these URLs to a Google Sheet like this, and then like this, and then Google Sheets should allow us to autocomplete this. Whoopsies. Yeah, so we can go all the way down to page 100.

At some point here, you know, you get the point and then, um, we can loop over these URLs instead of having to click the next page button. So what I'm going to do is I'm going to start a new automation. So I'll click new automation web, and then we don't need cookies.

And then I am going to enter the first URL here, and this is page one, and then we can scrape a list. Oh, we need to dismiss this pop up first.

because that's hidden behind, we can use suggested steps instead. So just click it once and then go ahead and approve this. Make sure that this is allow error so that if it doesn't show up, this doesn't cause any problems. Next we can use scrape list and then we can click the names and then we can click the who it's by.

You can see it looks like this is a little hidden. Um, the text is kind of hidden in this table, which is interesting. I'll check why that is, but we can grab all of the prices or whatever else we needed to here, and then we can also, um, grab the links. By duplicating that. Sorry about how hard that is to see.

It looks like there's something weird with the table being displayed on this website, but that won't affect it scraping. Um, but anyways, we can see all of the titles here and the URL for every shirt and then who it's by here. So I'll confirm that. And then this automation is basically done. We just need to set up using the list trigger.

So I'll click I'm done here. And then we need to share the Google sheet that we have with automations at taskmagic. com. Then what we can do is we can grab this Google sheet URL and change this to be a list trigger. And then we can say, use a list from a Google sheet and set up our sheet connection, and this is allowing us to now loop over all of these URLs.

So we can say that we want to do, um, 10 rows at a time, or maybe a hundred. I don't know what you want to do. If you're scraping this all in one go. And then we can change the first go to page step here to be a URL, the one from our Google sheet, instead of that first page every time. And this should be all the way slid like that.

And this is going to be that automation. So the only thing you're going to want to tweak here is depending on the information you want to scrape from this shirt, like the Uh, title, name, whatever you want to grab, maybe the image. Um, that's what you'll be able to scrape to this Google Sheet. Then what we can do is we can loop over this information if there's more that you want to do here, uh, after we scrape all of these shirts.

So to put all of the scrape data in a Google sheet, we can just make an output sheet or name this, whatever we want. We can add some headers here, like title author, or I guess whoever it owner is. And then try, or I think it was link and then author, we can set this up as who we're sending data to. So we click send to Google sheets, paste the spreadsheet, paste output, and then say, it looks good.

And now what this will do is this will export all of these sheets from here, wherever the funny page was that will export the name, the URL to the t shirt and then the owner right here to this Google sheet. If we wanted to get more information, like I'm saying, by, uh, basically clicking into every, um, and then scraping additional stuff here, you can repeat the same process looping over all of the links from the shirts that you scraped.