One of the greatest challenges in crawling for expired domain names, is knowing where to crawl.
You don’t need to know how to crawl, because this is taken care of by the expired domain crawler you’re using with PBN Lab, and while there are some variations we can implement, the process itself doesn’t vary greatly.
In this post I’m sharing with you the systematic approach I use to constantly reveal great expired domains.
This is almost literally a formula for finding domains.
And the best part is…you don’t need to think hard, you just need to fill in the blanks!
Why just 10 domains? Couldn’t I find more?
You’ll find thousands of domains. But the real question is, how many of those are worth keeping?
Each crawl reveals around 70 domains on average. Generally 3 to 5 of those are “good” domains, and 1 in 10 crawls will return a great domain.
You could use this method to find hundreds of expired domains, you just need to repeat this process over and over.
Aiming for 10 domains is realistic, you might find lots more. It depends on a lot on what you’re expecting in terms of metrics and other factors.
But by putting this systematic approach into action, it’s simply just a matter of putting the expired domain crawler to work, and let it do all the heavy lifting.
The real challenge is working out which domains to keep, and which to ignore!
A systematic approach to domain crawling
This is essentially a game, where you provide 5 easy answers to some relatively simple questions.
For this to work for you, all you’ll need is a little knowledge of your niche (one would hope you already have this!), or you’ll simply need to do a little Googling to fill in the blanks.
The answers you provide are blended together with some other search terms to form over 100 unique searches you can use to quickly generate seed URLs, and then put to use in the domain crawler.
I have found that this makes coming up with seed URLs a whole lot easier, and it lets you focus on the result, rather than on job creation. It means you can simply let the crawler do its thing.
Now, you might look at this and think this is far too simple. But the truth is, there’s no advantage in making this more complex than it needs to be.
Tip: You can also use this simple method for outreach. These searches reveal sites related to your niche. Why not network with other people, make blog comments, share your content and ask for backlinks too?
The keyword builder for PBN Lab:
There are 7 main topics I focus on to generate keyword phrases for crawling, which include “who, what, when, where, professions, communities and how“.
Below is a quick explanation of what each topic is about and some sample answers, using my cycling niche as an example.
Let’s get started – so you can see exactly how simple this is.
Who (famous people):
If you were to mention your niche to someone, who do they immediately associate with? Is it a famous scientist, a politician, actor, sportsperson, supermodel or a TV character perhaps?
Name 5 famous people related to your niche:
- Bradley Wiggins
- Cadel Evans
- Lance Armstrong
- Jens Voigt
- Chris Hoy
What (things or products):
Chances are there’s a physical thing related to your niche. For product-based niches, or ecommerce sites, this is dead simple. It’s the dumbbells, a baby pram or hiking boots.
Name 5 tangible things, or products related to your niche:
- Mountain bike
- Bicycle helmet
- Cycling shoes
- Bike tyres
- Bike Pump
When (events related to your niche):
Just like the “who” topic, there’s almost always an event you can relate to your niche.
Is it a sporting event, an annual expo or convention? Is it Christmas? Is it “tax time”? It could even be a TV show.
Name 5 events related to your niche:
- Tour de France
- Olympics cycling
- Mountain bike championships
- ITU World Triathlon
- UCI Road World
Where (a place in the world):
Are there places in the world that are synonymous with your niche?
Sometimes you’ll need to get creative with this and put in some extra thought, but there’s always somewhere in the world you can tie back to your niche. For instance, if your niche site is targeting new born mothers, it could be simply a hospital. If your niche is targeting cycling shoes, it could be France.
NB: Don’t get too bogged down on this. If you have no idea simply name places in the world that are perhaps in wealthy countries, and are high populated.
Where are 5 places in the world related to your niche that you can think of?
Alternatively, name 5 countries that are most relevant to your niche:
- United States of America
Profession (job titles):
No doubt someone is getting paid good money to serve customers in your niche.
If you think back to who we think of, what is their day-to-day job, and what do we refer to them as? Are they “professional cyclist”, “web developer” or “school teacher”?
Name 5 jobs that are specifically related to your niche, or are at least closely related to your niche:
- Professional cyclist
- Bike mechanic
- Cycling coach
- Bike fitter
Where are people meeting one another when they’re involved in this niche?
In every niche, there are people who want to associate with other people and share their common interest.
These communities are everywhere, in most towns and cities, and of course online.
Where are 5 places people in your niche meet up, or may bump into one another?
NB: If you can’t think of a “place”, you can also add a niche-specific activity. See number 5 as an example.
- Cycling club
- Ride group
- Spin classes
- Bike shops
- Cycling for the elderly
How (where does a newbie start?):
What’s the most obvious question you’d ask if you were new to this niche? If you were Googling this question, what would you write? How do I do ___? Where’s the best place to find ___? A beginners guide to ___. Getting started with ____.
What are questions that newcomers would have?
You can also simply name topics related to the niche, as I’ve done in answer number 5.
- How do I replace a bicycle tube?
- How high should my seat be?
- How to fit tubeless tyres
- Do I need clipless pedals?
- Stretches for cycling
That was easy! What do I do now?
As you can see, there’s nothing overly difficult about simply filling in the blanks.
But what exactly do you do with the answers? That’s equally as simple!
I’ve got a very basic Google Sheet setup, with a single worksheet for each topic.
All you need to do is put the answers in the appropriate cells each in each worksheet, and then on the last worksheet, you’ll see that formulas have done the rest of the work and we simply see the resulting keyword search terms.
Here’s a quick screenshot of what keyword terms my answers produce. NB: there are more results off-screen:
From here, the final step is to add each job to the crawler.
I highly recommend that you batch the jobs, adding ten or fifteen at a time. Let the jobs complete, review the results, then come back and do another ten. It means less overwhelm, and avoids the monotony of adding 100 plus jobs back-to-back!
Now for the money shot…
How to add your jobs to the crawler:
- Copy the keyword phrase from the cell in the Google sheet:
- Once you’ve logged in to PBN Lab, simply add a new job:
- Paste the keyword phrase as your Job Name
- Paste the keyword phrase in the Keywords field
- Wait for the Google SERPs to come back, then click Finish to add your job to the crawler.
- Click Add a New Job to start again from Step 1
And the results of my crawl?
As you can see by the steps above, it took me about ten seconds to create the first job – while the job completed in just 13 minutes.
This crawl returned 102 expired domains, and the top result (sorted by Trust Flow) was friendsofemigration.org.
It has a Trust Flow of 18, Citiation Flow of 14, Domain Authority of 10 and a Page Authority of 25.
Best of all, as you can see in the screenshot, it’s Trust Flow Topic is ‘Sports/Cycling’, which is spot on!
That’s not a bad domain, certainly not amazing, but given how much work I didn’t do to find it, I certainly cannot complain.
With more than 100 jobs I can now easily queue up, I’m sure to find as many domains as I need for this niche site.
How can I get my hands on the Google Sheet?
Now that you’re ready to perform your first crawls, and have a list of crawls to perform as long as your arm, what are you waiting for?
Every subscriber to PBN Lab is provided with this cheat sheet on sign up!
I want my domains now!
I just scoured your site and very interested by your service. I was almost decided to go forward and take the Tera Week option but I don’t know if AFTER the week, i can still have access to my account or the account is closed and I have to pay again to log in. Maybe you already answered this question ?
I understood that over the week I will not have option to make new job, this part is ok 😀 but what about the access to check expired found during the tera week paid ?
Good question! Yes, you still have full access to PBN Lab after the first week (or when any subscription period ends). The only difference is that the crawler(s) will not process your jobs after your subscription has expired.
Looking forward to having you on board.
P.s. My apologies for the very delayed response, I’ve not been getting notifications from WordPress on new comments of late. I best look into that! =)
Can you share this google sheet with me? :(((
Sure! The link is in your “welcome email” when you sign up. Haha 😉
Just to make sure, do I need to provide my own proxies to run pbnlab? if yes, how many proxies you recommend?
You don’t need to provide any proxies to run PBN Lab. This is a managed service “in the cloud”, so you just run it all from your web browser.
hi Scott, if I understand correctly, I can subscribe to your service for just one month till I find the domain I want and then cancel my subscription if I wish, correct? and I can still access the UI just without access to the crawler, correct?
That’s right, once you have a subscription set up, you can still work in PBN Lab after your subscription expires.
Hi Scott, I’ve just signed up for a monthly plan and run my first search. How can I exclude from it (not filtering in the results but in the step before) all the .edu, .gov or .mil domains? I am a private and I cannot register any of those and I’d like to make the search faster removing those that are useless for me. Thanks
Thanks for joining PBN Lab, it’s good to have you on board.
I’ve actually noticed a huge number of these TLDs (.mil, .gov, .gov.uk etc) coming through, and I’ve disabled them. Just about nobody can register these, so it’s a complete waste of time for all involved.
But you can actually set the TLD filters on this page within PBN Lab, which will allow you to narrow your list right down: https://my.pbnlab.com/tools/user/priority-tlds
Please be sure to send future requests like this to firstname.lastname@example.org so they don’t get overlooked!