Engineer
Super Moderator
- Joined
- Dec 14, 2006
- Messages
- 3,210
- Reaction score
- 400
Does anyone have a good tool or a good method for finding the underlying source data of a page that has javascript-generated content?
It's easier if I explain with an example.
Go to https://www.allslotscasino.com/slot-machines
View source (Control U), and then search (Control F) for the first slot game in the list: Immortal Romance.
Nothing...
The javascript that generates the page is getting it from somewhere -- the question is, where? Where is the javascript or JSON file that the list of games is coming from? Does anyone know of a good way to find where this content is? Not just for this site, but for any site I want?
Sometimes I can find what I am looking for by viewing the source code and finding an included file, or by using Chrome DevTools and looking at the network activity.
Why they are doing it this way instead of using regular HTML isn't clear to me. It is cloaking, for sure, but why they are doing it, I'm not quite sure. To speed up the page load time, maybe? Okay, but by doing this, Google (I think) can't see any of the game names, which will negate any benefit they might get by having a falsely-fast load time. The page loads quick, great, but there's nothing here worth seeing....
The reason I want to find the underlying data source is because it is far easier to run a daily script that collects all of the available game names, instead of manually putting them into my database.
Some sites have over 1,000 games from 30 different providers, and it isn't possible to manually keep track of the lists anymore.
Sometimes brands will add a game, keep it for a month (or sometimes literally one single day), and then remove it. *cough* bet365, Leo Vegas.
So, if anyone has tips on how to find the source, please let me know.
I have asked several programs for a "feed" of their games -- some do it, no problem, but others act like you are asking for the moon. Really it shouldn't be that difficult -- just use the same query you currently use to display the list of available games inside the casino, and publish it to a regular URL that shows the complete list.
Thanks in advance.
It's easier if I explain with an example.
Go to https://www.allslotscasino.com/slot-machines
View source (Control U), and then search (Control F) for the first slot game in the list: Immortal Romance.
Nothing...
The javascript that generates the page is getting it from somewhere -- the question is, where? Where is the javascript or JSON file that the list of games is coming from? Does anyone know of a good way to find where this content is? Not just for this site, but for any site I want?
Sometimes I can find what I am looking for by viewing the source code and finding an included file, or by using Chrome DevTools and looking at the network activity.
Why they are doing it this way instead of using regular HTML isn't clear to me. It is cloaking, for sure, but why they are doing it, I'm not quite sure. To speed up the page load time, maybe? Okay, but by doing this, Google (I think) can't see any of the game names, which will negate any benefit they might get by having a falsely-fast load time. The page loads quick, great, but there's nothing here worth seeing....
The reason I want to find the underlying data source is because it is far easier to run a daily script that collects all of the available game names, instead of manually putting them into my database.
Some sites have over 1,000 games from 30 different providers, and it isn't possible to manually keep track of the lists anymore.
Sometimes brands will add a game, keep it for a month (or sometimes literally one single day), and then remove it. *cough* bet365, Leo Vegas.
So, if anyone has tips on how to find the source, please let me know.
I have asked several programs for a "feed" of their games -- some do it, no problem, but others act like you are asking for the moon. Really it shouldn't be that difficult -- just use the same query you currently use to display the list of available games inside the casino, and publish it to a regular URL that shows the complete list.
Thanks in advance.