Quotes from Dan Kaminsky's Keynote at DEF CON China

Above is Dan Kaminsky's keynote at the inaugural DEF CON China.  It was nominally about Spectre and Meltdown, and I thought it was immediately applicable to testing at all levels.  Here are some moments that jumped out at me:

On Context:

"There's a problem where we talk about hacking in terms of only software...What does hacking look like when it has nothing to do with software." 1:55

"But let's keep digging." Throughout, but especially 5:40

"Actual physics encourages 60 frames per second. I did not expect to find anything close to this when I started digging into the number 60...This might be correct, this might not be. And that is a part of hacking too." 6:10

"Stay intellectually honest as go through these deep dives. Understand really you are operating from ignorance. That's actually your strong point. You don't know why the thing is doing what it is doing...Have some humility as you explore, but also explore." 7:40

"We really really do not like having microprocessor flaws...and so we make sure where the right bits come in, the right bits come out. Time has not been part of the equation...Security [re: Specter/Meltdown] has been made to depend on an undefined element. Context matters." 15:00

"Are two computers doing the same thing?...There is not a right answer to that. There is no one context. A huge amount of what we do in hacking...is we play contexts of one another." 17:50

[Re: Spectre and Meltdown] "These attackers changed time which in this context is not defined to exist...Fast and slow...means nothing to the chip but it means everything to the users, to the administrators, to the security models..." 21:00

"Look for things people think don't matter. Look for the flawed assumptions...between how people think the system works and how it actually does." 35:00

"People think bug finding is purely a technical task. It is not because you are playing with people's assumptions...Understand the source and you'll find the destination." 37:05

"Our hardest problems in Security require alignment between how we build systems, and how we verify them. And our best solutions in technology require understanding the past, how we got here." 59:50

On Faulty Assumptions:

"[Example of clocks running slow because power was not 60Hz] You could get cheap, and just use whatever is coming out of the wall, and assume it will never change. Just because you can doesn't mean you should...We'll just get it from the upstream." 4:15

"[Re: Spectre and Meltdown] We turned a stability boundary into a security boundary and hoped it would work. Spoiler alert: it did not work." 18:40

"We hope the design of our interesting architectures mean when we switch from one context to another, nothing is left over...[but] if you want two security domains, get two computers. You can do that. Computers are small now. [Extensive geeking out about tiny computers]" 23:10

"[RIM] made a really compelling argument that the iPhone was totally impossible, and their argument was incredibly compelling until the moment that Steve Jobs dropped an iPhone on the table..." 25:50

"If you don't care if your work affects the [other people working on the system], you're going to crash." 37:30

"What happens when you define your constraints incorrectly?... Vulnerabilities. ...At best, you get the wrong answer. Most commonly, you get undefined behavior which in the presence of hacking becomes redefinable behavior." 41:35

"It's important to realize that we are loosening the assumption that the developer knows what the system is supposed to do...Everyone who touches the computer is a little bit ignorant." 45:20

On Heuristics

"When you say the same thing, but you say it in a different time, sometimes you're not saying the same thing." 9:10

"Hackers are actually pretty well-behaved. When hackers crash code...it does really controlled things...changing smaller things from the computer's perspective that are bigger things from a human's perspective." 20:25

"Bugs aren't random because their sources aren't random." 35:25

"Hackers aren't modeling code...hackers are modeling the developers and thinking, 'What did [they] screw up?' [I would ask a team to] tell me how you think your system works...I would listen to what they didn't talk about. That was always where my first bugs came from." 35:45

On Bug Advocacy

"In twenty years...I have never seen stupid moralization fix anything...We're engineers. Sometimes things are going to fail." 10:30

"We have patched everything in case there's a security boundary. That doesn't actually mean there's a security boundary." 28:10

"Build your boundaries to what the actual security model is...Security that doesn't care about the rest of IT, is security that grows increasingly irrelevant." 33:20

"We're not, as hackers, able to break things. We're able to redefine them so they can't be broken in the first place." 59:25

On Automation

"The theorem provers didn't fail when they showed no leakage of information between contexts because the right bits went to the right places They just weren't being asked to prove these particular elements." 18:25

"All of our tools are incomplete. All of our tools are blind" 46:20

"Having kind of a fakey root environment seems weird, but it's kind of what we're doing with VMs, it's what we're doing with containers." 53:20

On Testing in the SDLC

"We do have cultural elements that block the integration of forward and reverse [engineering], and the primary thing we seem to do wrong is that we have aggressively separated development and testing, and it's biting us." 38:20

"[Re Penetration Testing]: Testing is the important part of that phrase. We are a specific branch of testers that gets on cooler stages...Testing shouldn't be split off, but it kinda has been." 38:50

Ctd. "Testing shouldn't be split off, but it kinda has to have been because people, when they write code, tend to see that code for what it's supposed to be. And as a tester, you're trying to see it for what it really is. These are two different things." 39:05

"[D]evelopers, who already have a problem psychologically of only seeing what their code is supposed do, are also isolated from all the software that would tell them [otherwise]. Anything that's too testy goes to the test people." 39:30

"[Re: PyAnnotate by @Dropbox] 'This is the thing you don't do. Only the developer is allowed to touch the code.' That is an unnecessary constraint." 43:25

"If I'm using an open source platform, why can't I see the source every time something crashes? ...show me the source code that's crashing...It's lovely." 47:20

"We should not be separating Development and Testing... Computers are capable of magic, and we're just trying to make them our magic..." 59:35

Misc

"Branch Prediction: because we didn't have the words Machine Learning yet. Prediction and learning, of course they're linked. Kind of obvious in retrospect." 27:55

"Usually when you give people who are just learning computing root access, the first thing they do is totally destroy their computer." 53:40 #DontHaveKids

"You can have a talent bar for users (N.B.: sliding scale of computer capability) or you can make it really easy to fix stuff." 55:10 #HelpDesk

"[Re: Ransomware] Why is it possible to have all our data deleted all at once? Who is this a feature for?!... We have too many people able to break stuff." 58:25

A) Original Twitter Thread
B) Thread Unroll from @threadunrollapp

Postman Masterclass Pt. 2

During my second Postman meetup as part of the Las Vegas Test Automation group, we were able to cover some of the more advanced features of Postman. It's a valuable tool for testing RESTful services (stronger opinions on that also exist), and they are piling on features so fast that it is hard to keep track. If you're a business trying to add automation, Postman is easily the lowest barrier to entry to doing so. And with a few tweaks (or another year of updates) it could probably solve most of your API testing.

The meetup covered the Documentation, Mock Server and Monitor functionality. These are pieces that can fit in your dev organization to smoothe adoption, unroadblock, and add automation with very little overhead. Particularly, the Mock servers they offer can break the dependency on third party integrations quite handily. This keeps Agile sprints moving in the face of outside roadblocks. The Monitors seem like a half-measure. They gave a GUI for setting up external monitors of your APIs, but you still need Jenkins and their Newman node package to do it within your dev env. The big caveat with each of these is that they are most powerful when bought in conjunction with the Postman Enterprise license.  Still, at $20 a head, it's far and away the least expensive offering on the market.

Since the meetup, I've found a few workarounds for the features I wish it had that aren't immediately accessible from the GUI. As we know in testing in general, there is no one-size fits all solution.  And the new features are nice, but they don't offer some of the basics I rely on to make my job easier.  Here is my ever-expanding list of add-ons and hidden things you might not know about.  Feel free to comment or message me with more:

Postman has data generation in requests through Dynamic Variables, but they're severely limited in functionality. Luckily, someone dockerized npm faker into a restful service. This is super easy to slip stream into your Postman Collections to create rich and real-enough test data. Just stand it up, query, save the results to global variables, and reuse them in your tests.

The integrated JavaScript libraries in the Postman Sandbox are worth a fresh look. The bulk of my work uses lodash, crypto libraries, and tools for validating and parsing JSON. This turns your simple requests to data validation and schema tracking wonders. 

• Have a Swagger definition you don't trust? Throw it in the tv4 schema validator. 
• Have a deep tree of objects you need to be able to navigate RESTfully? Slice and dice with lodash, pick objects at random, and throw it up into a monitor. Running it every ten minutes should get you down onto the nooks and crannies.

This article on bringing the big list of naughty strings (https://ambertests.com/2018/05/29/testing-with-naughty-strings-in-postman/amp/) is another fantastic way to fold in interesting data to otherwise static tests. The key is to ensure you investigate failures. To get the most value, you need good logs, and you need to pay attention to your results in your Monitors.

If you have even moderate coding skills among your testers, they can work magic on a Postman budget. If you were used to adding your own libraries in the Chrome App, beware: the move to a packaged app means you no longer have the flexibility to add that needed library on your own (faker, please?).

More to come as I hear of them.

27th Quantity Surveyors

The 27th Surveyors are part of the ground units on the death world Pholos IV.  They remove invasive vegetation from settlements, patrol for xenos incursion, and quell uprisings among the rowdy recolonists.

The deep green robes with shocking yellow lining mirror the vegetation that sprouts, grows tall, and flowers often in the same day. The temperate latitudes pulse with these colors when viewed from space.

The surveying contingent can rely on support from giant machines. Knights with claws and saws for felling overnight forests, Onagers specially modified to traverse the decaying swamps thick with fungal rot,  resurface roads thick with cracks only weeks after being layed down, and pulp the lignin and experimental protein bioheresy into specialized oils and unguents. The ruin wrought by the mad Biologis would be turned to the might of the Mechanicum at last.

Fixing Ford AC Head Controller Vacuum Problem

The AC on my land yacht (2009 Mercury Grand Marquis) has been in the fritz for a while. Last winter, it gradually stopped switching from max AC/recirculate (a necessary in Vegas), then got stuck on norm AC until it rested on Defrost/Floor. I was able to fix it with some basic troubleshooting, YouTube sleuthing, and two bucks in o-rings.

This shaky yet informative video by Ian Smith helped me diagnose it as a problem with vacuum only. The AC itself was fine. It blows cool air all day long. It just did so at the windshield. It couldn't be the blend-door actuator.

The same video showed me how to diagnose the vacuum problems. The black hose providing vacuum from the engine seemed fine: I was getting 20 inches of vacuum with the car turned on when I hooked up a bleed pump with a gauge (mine came from Harbor Freight, shown in the video). To test the actuators, all I had to do was hook a 'jumper' pipe from black to the other pipes. Each one seemed to hold air, and the actuators sprang to life once again. For the first time in a year, I had cold air blowing from the vents. The problem couldn't be in the lines. I pulled the controller head for a closer look.

The head itself is a bunch of electronics, a control panel, and one removable plate with four solenoids. The vacuum hoses come into this through a manifold, and the head controls trigger the solenoids to route vacuum from the black hose to the others. This triggers different actuators under the dash. Something was amiss in the manifold.

I returned to YouTube looking for rebuild instructions. I found this extremely helpful video from a Chicago mechanic. The solenoids contain an o-ring that dries out, wears out, and loses the ability to hold vacuum. I obtained close to the recommended o-rings from Lowes (#36, 5/16 OD, 3/16 ID, 1/16 thickness) as I was not willing to wait for Amazon. A little Oatey silicone lubricant made the tight squeeze work a little better. I found I had to seat the solenoid heads at least once before total reassembly. It was too difficult to do so at the end and fight with the other small parts at the same time. 45 minutes later, I had full control of my AC restored.

I can't believe it was this simple to fix the controller. I think I was intimidated by the AC (having spent $1500 last year to have the dealer redo the whole system from seals to refrigerant). I didn't want to break anything. A few targeted troubleshooting steps helped assuage any fears of irreparable harm, and now I have a comfortable cabin once again.

The Best Thing About Hero Quest...

...Are the Miniatures!

These were painted in the early 90s with my brother using craft paint while at my grandparents' cabin near Navajo Lake. They currently reside in the closet awaiting the cousins to kick in the doors with my son.

Apologies to BardicBroadcasts https://youtu.be/Cx8sl2uC46A

The Glowhawk: OFBC Gaiden

Update: Plume added, rebuilt using thread and pipe cleaners to keep it upright and separate strands, removed plasticard sticks and zip ties.  Also pulled the EL Wire which is being repurposed in my son's EL Hoodie.

We were only able to print a few OFBC 2.0 cases before DEFCON 26. The leftover parts would have sat in my toolbox for quite a while if not for a serendipitous mistake: I ordered the wrong color LEDs from Sparkfun. This plus a little construction advice from a seamstress helped me cobble together the glowing headgear that is The Glowhawk

My courage and thinning hair prevents me from getting a mohawk while at DEFCON, but I've always wanted one. Instead I started to create one with a networking theme. Pipe cleaners in the color of Cat6 twisted pair served as a thick mane anyone could be proud of. This was wired onto a hat as a test. It looked OK, but it was kind of stubby to wear all on its own.

The LED driver for the OFBC is overdone. A single charge can last 10 hours on the original model. I wondered how much it could handle in terms of output, and a little breadboarding showed me I could wire several of the LED modules together as long as they were in parallel. Now how to use them?

The LEDs are these 3W green modules with attached heat sink. Direct eye contact is not recommended (hence the pains we took to use momentary buttons on the OFBC). On the beer light, we diffused the over-bright light so it could be sculpted by the drink it passed through. I was inspired by a fiber optic dress I saw elsewhere and found fiber optic table centrepieces for dirt cheap on Amazon. Some hot glue joined the disassembled fiber optics to the bright LED. The mane of glowing green was born!

With this fresh take in hand, DEFCON was upon us. I packed my things and thought I might take a crack in the evening. The Richard Cheese show was the perfect venue to solder everything together. The_bozo and I found a better place to work where the hot glue gun could run safely. I transferred the existing Cat6 mohawk to a bright green John Cena hat from Goodwill. Inside the channel that ran between upturned pipe cleaners, I hot glued the modules and fiber optics. Zip ties kept the fiber bundles from flopping around too much.

I consider the Glowhawk a great success, if a tad impractical. It lasted about 2 hours on a charge, and I was able to walk around wearing it with the mobile party crew for that long before it got uncomfortable.  A photo of me wearing it hit the DEFCON Closing Ceremonies, and my son keeps trying to steal my remaining fiber optics for a lamp in his room.

Future improvements include better internal support, googly eyes to cover the logos, and a fifth plume to fill out the front. See you next year!

Urns

My father passed late last year, and I made three nondescript urns as keepsakes for family and friends. It was the first time I made a box of any respectability since 2000.  I hadn't originally planned to make them when he passed, but making them helped me process things in a difficult time.

I was the responsible party for my father's estate as his wife does not speak English very well. As such, it fell to me to arrange the funeral, notify friends, and start to organize his affairs. I kept it together. The arrangements were made, the bills were covered, and all in a few days. I kept it together, that is, until I tried to return to work. I got ready. I even got in my car to go. But I could not. Instead, I went into the shop and executed a simple design for holding a portion of his ashes.

The material is Indian Rosewood (the same that I used for the magnetic bottle openers). The strong grain made mitered corners a natural choice. I even had enough contiguous grain to try to book-end most sides. I didn't have a keyed or splined miter jig (which could have strengthened the corners), but I figured the lid and bottom would provide a good brace against failure.

Dimensioning the lumber wasn't very difficult; it was the geometry of the corners that caused me real trouble. I left the sides thick to give each box some heft. I eyeballed the lid thickness and shaved down some beautiful figured grain to just the right height (maybe I overshot it a little and had to clean it up later). When I got to cutting the miters, I found that I didn't have any accurate way to match them up. The miter saw was definitely not accurate from cut to cut. I lost a lot of material on the table saw trying to get a canted blade to just the right angle. I finally settled on using my miter sled. I had to cut the sides down a bit to make sure I could make the entire cut in one pass. By the end of this therapeutic day, I had three roughly identical boxes ready for glue-up.

The second half took a few more months to pull off. Uncertainty about the accuracy of the cuts lead me to put the project on hold. Should I delay and try to true then with a shooting board? My girlfriend gave me the most wonderful advice once: when you find yourself rushing a project, put it down and come back later. The parts to three urns marinated on the bench and in my mind for a few months.

A test fit in March didn't seem too bad. The time off convinced me to persevere and get them together. I discovered too late that I mixed up the orientation of the edges. My careful bookends were a jumble on two of the three boxes. However, the imperfect corners and dimensional problems worked to hide the errors amongst each other. Sanding trued up protruding tear-out and splinters without obvious rounded-off corners. Finally, dark stain and some paste wax finished the work of hiding imperfect joints in dark recesses and shiny polished surfaces.

I finished the bottom with plywood. If I had to pick a spot where I'm uncertain about my choices, it's here. Glue is strong, but how will the baltic birch bottom hold up over time? I'm thinking of throwing in some brads there just in case. The bottom served as a canvas whereon I could memorialize my father. I was able to burn the message "Invictus Maneo", the Armstrong Clan (and our ancestral) family motto. Loosely translated, it means, "I remain unconquered."

This entire project was an object lesson in how I'm still learning some of the most basic techniques in woodworking. I need a way to clean up miters that start on the saw. A shooting board or similar has been recommended. Fine adjustments on my existing miter sled might also work. Though it didn't seem too bad once finished, the tearout for certain cuts makes me think I have a dull blade. I'll have to investigate, tune, and try again.

I think I've worked through a phobia of complex geometry. Something my father always talked about is how to hide your mistakes in woodworking. Bookends, miters, and a fitted lid left precious room for that, but I found a few tricks along the way such as meticulous test fitting, blue tape as clamps for difficult pieces, and patience above all. Regardless, I'm looking forward to the next boxes I build. I hope those have a markedly different emotional footprint.