Puppeteer Go

I love Puppeteer – it lets me play around with the ideas of The Headless Web – that is running the web in a browser without a visible browser and even build tools like DOM-curl (Curl that runs JavaScript). Specifically I love scripting the browser to scrape, manipulate and interact with pages.
One demo I wanted to make was inspired by Ire’s Capturing 422 live images post where she ran a puppeteer script that would navigate to many pages and take a screenshot.


This content originally appeared on Modern Web Development with Chrome and was authored by Paul Kinlan

<p>I love Puppeteer - it lets me play around with the ideas of <a href="https://paul.kinlan.me/the-headless-web/">The Headless Web</a> - that is running the web in a browser without a visible browser and even build tools like <a href="https://paul.kinlan.me/domcurl/">DOM-curl</a> (Curl that runs JavaScript). Specifically I love scripting the browser to scrape, manipulate and interact with pages.</p> <p>One demo I wanted to make was inspired by Ire's <a href="https://bitsofco.de/how-i-created-488-live-images/">Capturing 422 live images</a> post where she ran a puppeteer script that would navigate to many pages and take a screenshot. Instead of going to many pages, I wanted to take many screenshots of elements on the page.</p> <p>The problem that I have with Puppeteer is the opening stanza that you need to do anything. Launch, Open tab, navigate - it's not complex, it's just more boilerplate than I want to create for simple scripts. That's why I created <a href="https://github.com/PaulKinlan/puppeteer-go">Puppeteer Go</a>. It's just a small script that helps me build CLI utilities easily that opens the browser, navigates to a page, performs <em>your</em> action and then cleans up after itself.</p> <p>Check it out.</p> <div class="highlight"><pre style="color:#f8f8f2;background-color:#272822;-moz-tab-size:4;-o-tab-size:4;tab-size:4"><code class="language-JavaScript" data-lang="JavaScript"><span style="color:#66d9ef">const</span> { <span style="color:#a6e22e">go</span> } <span style="color:#f92672">=</span> <span style="color:#a6e22e">require</span>(<span style="color:#e6db74">'puppeteer-go'</span>); <span style="color:#a6e22e">go</span>(<span style="color:#e6db74">'https://paul.kinlan.me'</span>, <span style="color:#a6e22e">async</span> (<span style="color:#a6e22e">page</span>) => { <span style="color:#66d9ef">const</span> <span style="color:#a6e22e">elements</span> <span style="color:#f92672">=</span> <span style="color:#a6e22e">await</span> <span style="color:#a6e22e">page</span>.<span style="color:#a6e22e">$$</span>(<span style="color:#e6db74">"h1"</span>); <span style="color:#66d9ef">let</span> <span style="color:#a6e22e">count</span> <span style="color:#f92672">=</span> <span style="color:#ae81ff">0</span>; <span style="color:#66d9ef">for</span>(<span style="color:#66d9ef">let</span> <span style="color:#a6e22e">element</span> <span style="color:#66d9ef">of</span> <span style="color:#a6e22e">elements</span>) { <span style="color:#66d9ef">try</span> { <span style="color:#a6e22e">await</span> <span style="color:#a6e22e">element</span>.<span style="color:#a6e22e">screenshot</span>({ <span style="color:#a6e22e">path</span><span style="color:#f92672">:</span> <span style="color:#e6db74">`</span><span style="color:#e6db74">${</span><span style="color:#a6e22e">count</span><span style="color:#f92672">++</span><span style="color:#e6db74">}</span><span style="color:#e6db74">.png`</span>}); } <span style="color:#66d9ef">catch</span> (<span style="color:#a6e22e">err</span>) { <span style="color:#a6e22e">console</span>.<span style="color:#a6e22e">log</span>(<span style="color:#a6e22e">count</span>, <span style="color:#a6e22e">err</span>); } } }); </code></pre></div><p>The above code will find the h1 element in my blog and take a screenshot. This is nowhere near as good as Ire's work, but I thought it was neat to see if we can quickly pull screenshots from canisuse.com directly from the page.</p> <div class="highlight"><pre style="color:#f8f8f2;background-color:#272822;-moz-tab-size:4;-o-tab-size:4;tab-size:4"><code class="language-JavaScript" data-lang="JavaScript"><span style="color:#66d9ef">const</span> { <span style="color:#a6e22e">go</span> } <span style="color:#f92672">=</span> <span style="color:#a6e22e">require</span>(<span style="color:#e6db74">'puppeteer-go'</span>); <span style="color:#a6e22e">go</span>(<span style="color:#e6db74">'https://caniuse.com/#search=css'</span>, <span style="color:#a6e22e">async</span> (<span style="color:#a6e22e">page</span>) => { <span style="color:#66d9ef">const</span> <span style="color:#a6e22e">elements</span> <span style="color:#f92672">=</span> <span style="color:#a6e22e">await</span> <span style="color:#a6e22e">page</span>.<span style="color:#a6e22e">$$</span>(<span style="color:#e6db74">"article.feature-block.feature-block--feature"</span>); <span style="color:#66d9ef">let</span> <span style="color:#a6e22e">count</span> <span style="color:#f92672">=</span> <span style="color:#ae81ff">0</span>; <span style="color:#66d9ef">for</span>(<span style="color:#66d9ef">let</span> <span style="color:#a6e22e">element</span> <span style="color:#66d9ef">of</span> <span style="color:#a6e22e">elements</span>) { <span style="color:#66d9ef">try</span> { <span style="color:#a6e22e">await</span> <span style="color:#a6e22e">element</span>.<span style="color:#a6e22e">screenshot</span>({ <span style="color:#a6e22e">path</span><span style="color:#f92672">:</span> <span style="color:#e6db74">`</span><span style="color:#e6db74">${</span><span style="color:#a6e22e">count</span><span style="color:#f92672">++</span><span style="color:#e6db74">}</span><span style="color:#e6db74">.png`</span>}); } <span style="color:#66d9ef">catch</span> (<span style="color:#a6e22e">err</span>) { <span style="color:#a6e22e">console</span>.<span style="color:#a6e22e">log</span>(<span style="color:#a6e22e">count</span>, <span style="color:#a6e22e">err</span>); } } }); </code></pre></div><figure><img src="https://paul.kinlan.me/images/2019-12-03-puppeteer-go-0.jpeg" alt="4.png"></figure> <figure><img src="https://paul.kinlan.me/images/2019-12-03-puppeteer-go-1.jpeg" alt="3.png"></figure> <figure><img src="https://paul.kinlan.me/images/2019-12-03-puppeteer-go-2.jpeg" alt="2.png"></figure> <figure><img src="https://paul.kinlan.me/images/2019-12-03-puppeteer-go-3.jpeg" alt="1.png"></figure> <figure><img src="https://paul.kinlan.me/images/2019-12-03-puppeteer-go-4.jpeg" alt="0.png"></figure> <p>Enjoy!</p>


This content originally appeared on Modern Web Development with Chrome and was authored by Paul Kinlan


Print Share Comment Cite Upload Translate Updates
APA

Paul Kinlan | Sciencx (2019-12-03T02:28:20+00:00) Puppeteer Go. Retrieved from https://www.scien.cx/2019/12/03/puppeteer-go-2/

MLA
" » Puppeteer Go." Paul Kinlan | Sciencx - Tuesday December 3, 2019, https://www.scien.cx/2019/12/03/puppeteer-go-2/
HARVARD
Paul Kinlan | Sciencx Tuesday December 3, 2019 » Puppeteer Go., viewed ,<https://www.scien.cx/2019/12/03/puppeteer-go-2/>
VANCOUVER
Paul Kinlan | Sciencx - » Puppeteer Go. [Internet]. [Accessed ]. Available from: https://www.scien.cx/2019/12/03/puppeteer-go-2/
CHICAGO
" » Puppeteer Go." Paul Kinlan | Sciencx - Accessed . https://www.scien.cx/2019/12/03/puppeteer-go-2/
IEEE
" » Puppeteer Go." Paul Kinlan | Sciencx [Online]. Available: https://www.scien.cx/2019/12/03/puppeteer-go-2/. [Accessed: ]
rf:citation
» Puppeteer Go | Paul Kinlan | Sciencx | https://www.scien.cx/2019/12/03/puppeteer-go-2/ |

Please log in to upload a file.



There are no updates yet.
Click the Upload button above to add an update.

You must be logged in to translate posts. Please log in or register.

Related Posts