In the world of automation, leveraging custom code can sometimes be the key to unlocking challenging tasks. While it's not always the go-to solution, custom JavaScript can play a vital role in enhancing your automation workflows. In this blog post, we'll explore the story of using custom JavaScript to tackle a specific automation task involving scraping data from Twitter profiles and capturing post links.

Our automation journey begins with a common scenario - scraping valuable information from Twitter profiles, including engagement metrics and post links. However, extracting specific data like post links proved to be a daunting challenge through conventional means.

Clicking through the familiar three dots on Twitter posts, followed by selecting the "embed post" option, seemed like a simple task. However, capturing the resulting link directly from the URL bar posed a significant hurdle due to the limitations of traditional scraping methods.

To overcome this obstacle, a custom code step was introduced into the automation workflow. By integrating a snippet of custom JavaScript, the automation process gained the ability to retrieve the complete URL directly from the browser window.

The custom JavaScript snippet deployed in this scenario was tailored to extract the precise URL required for the automation task. This innovative approach showcased the flexibility and power of incorporating custom code into automation workflows, providing a workaround for challenges that standard automation tools struggled to address.

While the use of custom JavaScript in automation may not always be necessary, its impact can be transformative when faced with complex tasks that demand a more tailored solution. By embracing the versatility of custom code, automation enthusiasts can enhance the efficiency and effectiveness of their workflows, opening up new possibilities for automation success.

The journey of integrating custom JavaScript into automation not only solved a specific challenge but also demonstrated the ingenuity and adaptability of automation enthusiasts. By exploring unconventional solutions and embracing the flexibility of custom code, we can push the boundaries of automation capabilities and unlock new opportunities for productivity and innovation.

Step 1-Click on three dots—Click on Embed post

Step 2- Copy the Link and Lick on Add button

Step 3-Add the link or edit - Click on Save

So using custom code in your automation, although ideally it's super rare, um, is definitely doable and has its purpose. So in this automation, I go to someone's Twitter profile. Um, I'm scraping some information about them and like their engagement and stuff. And then I'm clicking, uh, on the embed post section of their tweets.

So like every tweet will have like three dots and then you can click embed post. And I wanted to capture that link. So I was having a really difficult time using scrape link to capture this. Let me quickly pull this up to show you.

Okay. So I wanted to click these three dots and then click embed post. And then I wanted to grab this link. from the URL bar. And since we don't have a scrape URL or anything like that from the URL bar, only from the page, what I did is I added a custom code step. And in this custom code step, it just returns the window.

Um, this code specifically is returning the entire URL that we have here. So again, a workaround where I use custom JavaScript ideally doesn't need to be used, um, but definitely doable and can help your automations.