I scrape pages for one of my projects, and the best way to get around anti-scraping is to occasionally fill out captchas’ / do some browsing by hand to get through the testing (usually works for a good while).
Anyway I ran into a heck of a rabbit hole actually trying to tell selenium to use a specific profile, but then I finally ran into a post that solved it for me.
There are two options you need to use a custom profile (or even the default profile).
The key is the
user-data-dir is specifying where the profiles are stored on the disk, not a path to the profile but a path to the directory that has the profiles.
--profile-directory settings is saying “use this profile” in that directory. So if your profile as re stored at
/home/<userid>/.config/google-chrome then you would set
If the profile you wanted to use was the main one you would set
You can see what your profile directory is by going to
chrome:://version. There is a “Profile Path” variable, just make sure to remove the profile name from the path, and no trailing slash.
This is what my chrome options array looks like (ignore that I am using a custom class to init this as I have a bank of machines used for selenium and I do not have grid setup).