How To Use Web Scraping For SEO – 4 SEO Hacks

Hi Guys, my name is Arpit, co-founder NSDM – We provide courses in marketing and tech space. Today I’ll show you How To Use Web Scraping for SEO and 4 SEO hacks.

These are amazing hacks that’ll blow your mind away.

I bet you never thought of using web scraping for SEO this way, watch the video till the end to learn How To Use Web Scraping for SEO.

I’ll be using my favourite web scraping tool Octoparse for this video.

You must download it to perform these hacks the link is in the description.

Let’s begin the video

Hack Number 1 – Scrape Authors Data For Guest Blogging

So you need guest bloggers for your blog?

All you need to do find the best blog in your niche I am sure a lot of guest bloggers will be contributing as they need exposure.

Like if you are in SEO niche, one of the best blogs is Moz and as you can see they have so many guest bloggers contributing here.

You will easily find their contact information here.

We can scrape all of them using Octoparse and then do an email outreach inviting these authors to contribute in your blog.
You should offer them something in return though.

This hack could also be used for lead generation if these authors are your target audience.
Let me quickly show you how you can scrape them in Octoparse.

Once you have downloaded and create your account on Octoparse hover on new and click on advance mode.

Enter the website you want to scrape. Like for this video I will put www.moz.com/blog. Hit save.

Click on the author name and then click on select all.

If it doesn’t scrape all them then click on more author names that’s how you need to train it.

Like you can see now it has selected all 14 authors

Then click on extract the URL of selected element.

Then click on paginate to scrape more pages, then click on the next page button and then hit confirm.

You can see it is now on the 2 page of the blog and it has scraped all 14 links of the authors here.

That’s all for this task. Hit run, then run task on this device.

You will see a new window will pop up and it will start scraping.

Once you are done click on stop run.

Export the data, remove duplicates as the same author has contributed multiples times, save the excel file and then open it.

You can see the author links on which we can find their contact information.

So let’s make another task in Octoparse to scrape their contact information.

Hover on new and then select advance mode.

Click on import from file, select the excel file we just saved with authors links, select sheet 1, then select column as A1 and hit save.

Now keep selecting all that you want.

Since I want this entire data in the table I will click on table body.

Select as much data as available you can use this to personalize the email when you do the outreach campaign.

Then click on extract data.

Let’s go on another author’s page to check if this is working fine.

From the loop URLs step click on the list icon and then select any other link, then click on go to web page step.

Now we can see it has extracted everything we want.

So click on run, then run task on your device.

Now you can see it has started scraping.

Once you have enough data you can stop the run, export data and this is how your excel file will look like.

Hack Number 2 – Scrape Commenters To Promote Your Content.

You can scrape people who have commented on a blog that is similar to your topic.

Then you can do an email outreach to promote your content.

If your content is truly amazing you can gain blog subscribers with this hack.

Like if I have written a blog about on-page SEO. I will search a blog on the same topic on Google.

Like here I see backlinko ranking on the keyword on-page SEO.

Let’s open the blog, get down to the comment section.

Now we can see most of these people who have commented have linked to their websites.

The link is there in their name.

So what we can do is, we can scrape their name and their website from this page.

Then we can get their contact details from their website all on automation it is pretty simple to do.

Let’s first use Octoparse to scrape their websites.

So get into the advance mode and paste the link of the blog from where you want to scrape the commenters. Then click on save.

Let’s scroll to the bottom of the blog.

Click on the name of the commenter and then click on select all.

You will see Octoparse is smart enough to extract all commenters with their websites. Like we have found 100 here.

Then click on extract both text and the URL of the selected element.

That’s all just hit run, the run task on your device.

You will see it working in a new window.

Once finished click on export data, save the excel file and then open it.

So now we have their websites, to get their contact details we will use another tool called Phantombuster

They have a phantom called data scraping crawler I have linked to it through my blog which is in the description.

Just give it a list of websites and it will try to find and scrape Emails, Facebook Urls, Instagram Urls, Linkedin Urls, Phone Numbers, Twitter Urls and YouTube Urls

You can use any 1 phantom for free, the documentation is here on how to use it, and it’s pretty simple to do.

Hack Number 3 – Improve CTR

Now this one is kind off a black hat.

I just want to put this idea across not necessarily recommending you to do this.

Let’s discuss this in the comments.

So CTR is an important ranking factor for Google.

What if we manipulate it.

We can easily create a web scraping bot which can put your target keyword on Google and then click on your website even if it is on the second page, then spend some time, visit multiple pages on your website so it doesn’t affect your bounce rate negatively.

Imagine you schedule your bot to do this many times throughout the day with proxy IP rotation so you don’t raise any red flags.

Remember when Rand Fishkin the wizard of Moz conducted this experiment wherein he told his Twitter followers to write a certain query on Google and then click on a website that ranks on the 2nd page.

After his followers did that the website started ranking on the 1st page.

We are using the same logic here instead of people we can use bots.

Let me know what you think about this hack.

If somebody tries this please let me know how it turned out for you.

If you see any flaws in this hack would love to discuss that with you guys.

Hack Number 4 – Scrape Q/A Sites For Content Ideas And Backlinks

We can easily scrape question answer sites or forums like Quora, Reddit or Stack Overflow for generating content ideas.

When you create a content piece it is necessary to check such sites so you know what queries people have regarding the topic you want to cover.

You should try to answer as many questions as possible through your content.

Once your content is ready answer these questions on these platforms and link back to your content piece.

Let me quickly show you How To Use Web Scraping on Quora through Octoparse.

Hover on new and then select advanced mode.

Entre site as www.quora.com hit save.

We need to make it login first.

So click on the email field, then click on entre text and put in your email, hit confirm.

Then select the password field, then click on entre text and put in your password, hit confirm.

Then click on the login button and select click button.

Then add a step open page.

Now we want to add the page here that we want to scrape.

So head on to Quora, put in your keywords in the search bar.

Like my niche is AI for marketing so put in that, on this page Quora will show you all questions people have asked regarding this topic.

So we will copy the link of this page, go back to Octoparse, get into the action settings of the step go to web page and paste the URL.

Then get into after loading page, check scroll down the page after it is loaded, then select to the bottom of the page, under repeat I will put 200 if there are too many questions on the page you could increase this number, under wait time I will select 3 seconds and hit ok.

Then click on auto detect web page data.

You will see it has scraped some fields but we only need the first to columns so I will delete the rest.

You will see it has automatically added a page scroll, click on edit, under repeat put 200 and increase the wait time to 3 seconds, hit confirm.

Uncheck click on load more button because there is no load more button.

Then click on create workflow.

That’s all now we will click on run, then run task on your device.

You will see it has started doing its job.

I’ll fast forward the video. When the job was done I was able to scrape all questions with links regarding my topic.

That will be all for this video.

By the way Octoparse is coming up with black Friday sale they will offer a massive discount starting from 16th November make use of it if you are looking to buy the premium version.

If you learnt something new make sure you smash that like button.

Also subscribe so you don’t miss my new videos.

Thank you so much for watching.

See you in the next one

We cover skill like these in our AI for marketing course. Check it out