-
Notifications
You must be signed in to change notification settings - Fork 2
Expand file tree
/
Copy pathindex.html
More file actions
465 lines (423 loc) · 16 KB
/
index.html
File metadata and controls
465 lines (423 loc) · 16 KB
1
2
3
4
5
6
7
8
9
10
11
12
13
14
15
16
17
18
19
20
21
22
23
24
25
26
27
28
29
30
31
32
33
34
35
36
37
38
39
40
41
42
43
44
45
46
47
48
49
50
51
52
53
54
55
56
57
58
59
60
61
62
63
64
65
66
67
68
69
70
71
72
73
74
75
76
77
78
79
80
81
82
83
84
85
86
87
88
89
90
91
92
93
94
95
96
97
98
99
100
101
102
103
104
105
106
107
108
109
110
111
112
113
114
115
116
117
118
119
120
121
122
123
124
125
126
127
128
129
130
131
132
133
134
135
136
137
138
139
140
141
142
143
144
145
146
147
148
149
150
151
152
153
154
155
156
157
158
159
160
161
162
163
164
165
166
167
168
169
170
171
172
173
174
175
176
177
178
179
180
181
182
183
184
185
186
187
188
189
190
191
192
193
194
195
196
197
198
199
200
201
202
203
204
205
206
207
208
209
210
211
212
213
214
215
216
217
218
219
220
221
222
223
224
225
226
227
228
229
230
231
232
233
234
235
236
237
238
239
240
241
242
243
244
245
246
247
248
249
250
251
252
253
254
255
256
257
258
259
260
261
262
263
264
265
266
267
268
269
270
271
272
273
274
275
276
277
278
279
280
281
282
283
284
285
286
287
288
289
290
291
292
293
294
295
296
297
298
299
300
301
302
303
304
305
306
307
308
309
310
311
312
313
314
315
316
317
318
319
320
321
322
323
324
325
326
327
328
329
330
331
332
333
334
335
336
337
338
339
340
341
342
343
344
345
346
347
348
349
350
351
352
353
354
355
356
357
358
359
360
361
362
363
364
365
366
367
368
369
370
371
372
373
374
375
376
377
378
379
380
381
382
383
384
385
386
387
388
389
390
391
392
393
394
395
396
397
398
399
400
401
402
403
404
405
406
407
408
409
410
411
412
413
414
415
416
417
418
419
420
421
422
423
424
425
426
427
428
429
430
431
432
433
434
435
436
437
438
439
440
441
442
443
444
445
446
447
448
449
450
451
452
453
454
455
456
457
458
459
460
461
462
463
464
465
<!DOCTYPE html>
<html lang="en">
<head>
<title>Visual Regression Testing with BackstopJS</title>
<meta charset="utf-8">
<meta http-equiv="x-ua-compatible" content="ie=edge">
<meta name="viewport" content="width=device-width, initial-scale=1">
<link rel="stylesheet" type="text/css" href="pantheon-theme/styles/screen-16x10.css">
<!--<link rel="stylesheet" href="node_modules/shower-ribbon/styles/screen-16x10.css">
-->
</head>
<body class="shower list no-page-numbers">
<header class="caption">
<h1>Visual Regression Testing with BackstopJS</h1>
<p>Brought to you by the folks at Pantheon</p>
</header>
<section class="slide" id="cover">
<div class="title">
<div class="background"></div>
<div class="text">
<h2>Visual Regression Testing with BackstopJS</h2>
<!-- <p class="setup-instructions">SETUP: <a href="http://bit.ly/visual-regression-setup">bit.ly/visual-regression-setup</a></p> -->
</div>
</div>
<figure>
<img class="cover" src="pantheon-theme/pictures/pantheon-office.jpg" alt="The Pagoda atop the Pantheon office">
<figcaption class="white">
<a href="http://www.twitter.com/getpantheon">@getpantheon</a>
</figcaption>
</figure>
</section>
<section class="slide bio image-r">
<div class="text">
<h2>Hi, I'm David Needham</h2>
<p>Developer Advocate at Pantheon</p>
<p><strong>davidneedham</strong> on <a href="https://github.com/davidneedham/">GitHub</a> and <a href="https://www.twitter.com/davidneedham">Twitter</a></p>
<p><a href="mailto:david.needham@pantheon.io">david.needham@pantheon.io</a></p>
<p>In my free time I like:</p>
<ul>
<li>Biking with my wife and kids</li>
<li>Board games: <a href="http://is.gd/davidsgames">is.gd/davidsgames</a></li>
<li>Volunteering in the community</li>
</ul>
<figure>
<img class="cover" src="pictures/02-David-Needham-Pantheon.jpg" alt="David Needham">
</figure>
</div>
</section>
<!-- <section class="slide bio image-r">
<div class="text">
<h2>Hi, I'm Dan Ficker</h2>
<p>Customer Success Engineer at Pantheon</p>
<p><strong>dandaman</strong> on <a href="https://www.drupal.org/u/dandaman">Drupal.org</a> and <strong>deliriousguy</strong> on <a href="https://twitter.com/deliriousguy">Twitter</a></p>
<p><a href="mailto:daniel.ficker@pantheon.io">daniel.ficker@pantheon.io</a></p>
<p>Some things I enjoy, aside from coding:</p>
<ul>
<li>Listening to music and live music</li>
<li>Tech gadgets and the latest software</li>
<li>Movies of all types</li>
</ul>
<figure>
<img class="cover" src="pictures/03-Dan-Ficker.jpg" alt="Dan Ficker">
</figure>
</div>
</section> -->
<section class="slide">
<h2>Agenda</h2>
<table>
<tr><td>Intro and Setup</td></tr>
<tr><td>Visual Regression Testing</td></tr>
<tr><td>Using BackstopJS</td></tr>
<tr><td>Continuous Integration</td></tr>
<tr><td>Standalone Node App</td></tr>
<tr><td>Q&A + Extracurricular</td></tr>
</table>
</section>
<section class="slide">
<h2>Setup</h2>
<strong>You need:</strong>
<ul>
<li><a href="https://docs.npmjs.com/getting-started/installing-node">Node.js and npm</a> installed globally</li>
<ul>
<li>Windows users also need <a href="https://www.python.org/downloads/">Python</a> installed</li>
</ul>
<li><a href="https://github.com/garris/BackstopJS">BackstopJS</a> installed globally<ul><li><code>npm install -g backstopjs</code></li></ul></li>
<li>The <a href="https://github.com/davidneedham/visual-regression-testing-workshop"> Node.js example app</a></li>
<li>A “live” website that is web-accessible</li>
<li>A “dev” version of that same website (local is great)</li>
</ul>
</section>
<section class="slide subtitle">
<div class="title">
<div class="background"></div>
<div class="text one-line">
<h2>Visual Regression Testing</h2>
</div>
</div>
<figure>
</figure>
</section>
<section class="slide">
<h2>What is Visual Regression?</h2>
</section>
<section class="slide">
<h2>re·gres·sion</h2>
<p><strong>/rəˈɡreSH(ə)n/</strong></p>
<p><em>noun</em></p>
<ol><li>a return to a former or less developed state.</li></ol>
<br/>
<h3 class="next">When have you been plagued by Visual Regression?</h3>
<ol class="next">
<li class="next">Routine security updates and patches</li>
<li class="next">Theming and site building</li>
</ol>
</section>
<section class="slide image-r">
<h2>Killer Use Case #1: Routine Updates</h2>
<figure>
<img src="pictures/08-WordPress-updates-available-cropped.jpg" alt="A screenshot of a WordPress website with updates available">
</figure>
</section>
<section class="slide screenshot">
<h2>Live (before update):</h2>
<figure>
<img src="pictures/11-Manual-Visual-QA-Live-Desktop.gif" alt="An animated gif of a website before a change">
</figure>
</section>
<section class="slide screenshot">
<h2>Dev (after update):</h2>
<figure>
<img src="pictures/12-Manual-Visual-QA-Dev-Desktop.gif" alt="An animated gif of a website after a change">
</figure>
</section>
<section class="slide image-r">
<h2>Side by side:</h2>
<figure>
<img src="pictures/13-Manual-Visual-QA-side-by-side-700px.gif" alt="An animated gif of the website before and after the change side-by-side">
</figure>
</section>
<section class="slide image-r">
<h2>Side by side (mobile):</h2>
<figure>
<img src="pictures/14-Manual-Visual-QA-Dev-Mobile-Side-By-Side-700px.gif" alt="An animated gif of the website on mobile before and after the change side-by-side">
</figure>
</section>
<section class="slide image-r">
<h2>Difference:</h2>
<figure>
<img src="pictures/15-Automated-Visual-QA-Scrubber-900px.gif" alt="A screenshot of a website showing the differences highlighted">
</figure>
</section>
<section class="slide image-r">
<div class="text">
<h2>Visual Regression: Benefits</h2>
<ul>
<li class="next">"Reference vs Test" Comparison</li>
<ul>
<li class="next">Reference (production)</li>
<li class="next">Test (environment to compare)</li>
</ul>
<li class="next">Support for multiple viewports</li>
<li class="next">Easily scan your entire site</li>
<li class="next">Helpful error reporting</li>
<li class="next">Easy to write tests</li>
</ul>
</div>
<figure>
<img class="cover" src="pictures/16-Responsive_Web_Design_for_Desktop,_Notebook,_Tablet_and_Mobile_Phone.png" alt="A website on multiple devices">
</figure>
</section>
<section class="slide image-r">
<div class="text">
<h2>Limits to Visual Regression</h2>
<ul>
<li class="next">Lots of noise in the testing process</li>
<li class="next">High number of false negatives</li>
<li class="next">Difficulties with dynamic content</li>
<li class="next"><em>Still saves time</em></li>
</ul>
</div>
<figure>
<img class="cover" src="pictures/17-noisy-texture-300x300-o100-d11-c-ffffff-t0.jpg" alt="Static noise">
</figure>
</section>
<section class="slide subtitle">
<div class="title">
<div class="background"></div>
<div class="text one-line">
<h2>Using BackstopJS</h2>
</div>
</div>
<figure>
<img class="cover" src="pictures/22-backstop-bg.jpg" alt="The BackstopJS logo on a background">
</figure>
</section>
<section class="slide">
<h2>BackstopJS</h2>
<ul>
<li class="next">Render with headless Chromey or Puppeteer</li>
<li class="next">Simulate user interactions with simple JS</li>
<li class="next"><strong>Browser reports with visual diffs</strong>, CLI reports and JUnit reports</li>
<li class="next">Plays nice with CI and source control</li>
<li class="next">Run globally or locally as a <strong>standalone packaged app with Node</strong></li>
<li class="next">Incredibly easy to use: just 3 commands go a long, long way!</li>
</ul>
<p><a href="https://github.com/garris/BackstopJS">github.com/garris/BackstopJS</a></p>
</section>
<section class="slide code-block image-r">
<h2>Everything defined in the backstop.json file</h2>
<pre class="smaller"><code>
{
"id": "backstop_default",
<mark>"viewports": [</mark>
{
"label": "phone",
"width": 320,
"height": 480
},
{
"label": "tablet",
"width": 1024,
"height": 768
}
],
<mark>"scenarios": [</mark>
{
<mark>"label": "Homepage",</mark>
<mark>"url": "https://pantheonweeklydemosite.lndo.site/",</mark>
<mark>"referenceUrl": "https://www.pantheondemo.com/",</mark>
"readyEvent": "",
"readySelector": "",
"delay": 0,
"hideSelectors": [],
"removeSelectors": [],
"hoverSelector": "",
"clickSelector": "",
"postInteractionWait": 0,
"selectors": [],
"selectorExpansion": true,
"expect": 0,
"misMatchThreshold" : 0.1,
"requireSameDimensions": true
}
],
"paths": {
"bitmaps_reference": "backstop_data/bitmaps_reference",
"bitmaps_test": "backstop_data/bitmaps_test",
"engine_scripts": "backstop_data/engine_scripts",
"html_report": "backstop_data/html_report",
"ci_report": "backstop_data/ci_report"
},
"report": ["browser"],
"engine": "puppeteer",
"engineOptions": {
"args": ["--no-sandbox"]
},
"asyncCaptureLimit": 5,
"asyncCompareLimit": 50,
"debug": false,
"debugWindow": false
}
</code></pre>
</section>
<section class="slide">
<div class="text">
<h2>Running BackstopJS globally</h2>
<ol>
<li>Navigate to your local project</li>
<li>Initialize Backstop: <code>backstop init</code></li>
<li>Edit your <code>backstop.json</code> file:
<ul>
<li><code>label</code>: name of your scenario</li>
<li><code>url</code>: <em>local</em> or <em>development</em> URL</li>
<li><code>referenceUrl</code>: <em>production</em> URL</li>
</ul></li>
<li>Create new reference screenshots: <code>backstop reference</code></li>
<li>Run tests: <code>backstop test</code></li>
</ol>
</div>
</section>
<section class="slide image-r">
<div class="text">
<h2>Killer Use Case #2: Theming & Site Building</h2>
<blockquote class="twitter-tweet" data-lang="en"><p lang="en" dir="ltr">> It'll just be a quick fix<br><br>Narrator: It wasn't.</p>— I Am Devloper (@iamdevloper) <a href="https://twitter.com/iamdevloper/status/1027486050497773568?ref_src=twsrc%5Etfw">August 9, 2018</a></blockquote>
<script async src="https://platform.twitter.com/widgets.js" charset="utf-8"></script>
</div>
<figure>
<img class="cover" src="pictures/19-oh-no.jpg" alt="Boy with hands over his eyes">
</figure>
</section>
<section class="slide">
<div class="text">
<h2>Running BackstopJS (notes)</h2>
<ul>
<li>What happens if we change <code>viewports</code> or <code>scenarios</code>?</li>
<li>Regenerate references with <code>backstop reference</code></li>
<li>Rerun the test with <code>backstop test</code></li>
<li><strong>Gotcha:</strong> Is your updated site different than production?</li>
</ul>
</div>
</section>
<section class="slide subtitle">
<div class="title">
<div class="background"></div>
<div class="text one-line">
<h2>Continuous Integration</h2>
</div>
</div>
</section>
<section class="slide screenshot">
<figure>
<img class="cover" src="pictures/40-Automated Testing for EDU.jpg" alt="Robots serving platters of food and drink with the quote, What parts of your job can robots do better and faster than you?">
</figure>
</section>
<section class="slide image-r">
<div class="text">
<h2>Example #1: On Cron</h2>
<strong>Pantheon Example Terminus Auto Update Script</strong>
<ul>
<li class="next">Check for updates on cron</li>
<li class="next">Generate a new environment</li>
<li class="next">Apply updates and test</li>
<li class="next">Make backups everywhere</li>
<li class="next">Apply or notify you</li>
</ul>
<p><a href="https://github.com/pantheon-systems/example-terminus-auto-update-script">Get the script</a></p>
</div>
<figure>
<img class="cover" src="pictures/45-Pantheon-Example-Terminus-Auto-Update-Script-on-Github.jpg" alt="A screenshot of the github repo for Pantheon Example Terminus Auto Update Script">
</figure>
</section>
<section class="slide image-r">
<div class="text">
<h2>Example #2: On Commit</h2>
<strong>Automated Workflows in Drupal 8 with GitHub, Composer and CircleCI</strong>
<ul>
<li class="next">Add a module with Composer</li>
<li class="next">Push the branch & make a PR</li>
<li class="next">Build the site and run tests</li>
<li class="next">Notify you on Slack along the way</li>
</ul>
<p><a href="https://github.com/pantheon-systems/terminus-build-tools-plugin">Get the script</a></p>
</div>
<figure>
<a href="https://events.drupal.org/minneapolis2020/training/automated-workflows-drupal-8-github-composer-and-circleci"><img class="cover" src="pictures/45-drupalcon.jpg" alt="DrupalCon Seattle 2019"></a>
</figure>
</section>
<section class="slide subtitle">
<div class="title">
<div class="background"></div>
<div class="text one-line">
<h2>Standalone App</h2>
</div>
</div>
<figure>
</figure>
</section>
<section class="slide image-r">
<div class="text">
<h2>Clone the example repo</h2>
<p>We've created <a href="https://github.com/davidneedham/visual-regression-testing-workshop">an example repo</a> that we're going to clone to give you a jumpstart.</p>
<p>There are detailed instructions in the README to run through this on your own next time. </p>
<p>Today we're going to get started by cloning the repo:</p>
</div>
<code class="block">git clone https://github.com/davidneedham/visual-regression-testing-workshop</code>
<figure>
<img class="cover" src="pictures/24-screencapture-github-davidneedham-Visual-Regression-Testing-BackstopJS-demo-2018-05-30-19_02_53.jpg" alt="A screenshot of the repo we're cloning">
</figure>
</section>
<section class="slide">
<div class="text">
<h2>Running the example node app</h2>
<ol>
<li>Change into the folder <br /><code>cd visual-regression-testing-workshop</code></li>
<li>Checkout a new branch <br /><code>git checkout -b my-hacking-1</code></li>
<li>Install Node dependencies <br /><code>npm install</code></li>
<li>Run the example script <br /><code>npm run start</code></li>
<li>Wait for the results (it should open in your browser automatically)</li>
</ol>
</div>
</section>
<section class="slide">
<div class="text">
<h2>Customizing the example</h2>
<ul>
<li>Edit <code>inc/sitesToTest.js</code> is the list of sites to test</li>
<ul>
<li>Try adding a section for your own site.</li>
<li><code>nonProductionBaseUrl</code> is your local / development URL</li>
<li><code>productionBaseUrl</code> is your production site URL</li>
<li><code>pathsToTest</code> is the array of URIs to test for each site</li>
</ul>
<li>Edit <code>inc/backstopConfig.js</code> (the viewports, delay, etc)</li>
<li>Re-run the demo with <code>npm run start</code></li>
</ul>
</div>
</section>
<section class="slide image-r">
<div class="text">
<h2>WebOps</h2>
<p>WebOps is a set of practices that facilitates collaboration and automates processes to improve the productivity of the whole web team from developers and designers to content editors, stakeholders, and more.
</p>
<p>The result is cross-functional web teams empowered to develop, test, and release website changes faster and more reliably.</p>
</div>
<figure>
<img class="cover" src="pictures/46-webops-3.gif" alt="WebOps with Pantheon">
</figure>
</section>
<section class="slide subtitle">
<div class="title">
<div class="background"></div>
<div class="text one-line">
<h2>Q&A</h2><p>Let's build something!</p>
</div>
</div>
</section>
<footer class="badge">
<!--<a href="https://www.twitter.com/davidneedham">Tweet @DavidNeedham</a>-->
</footer>
<div class="progress"></div>
<script type="application/javascript" src="shower/shower.min.js"></script>
</body>
</html>