Provisioning VMware Workstation Machines from Artifactory with Vagrant

I wrote a small
Vagrantfile
and helper library for provisioning VMware VMs from boxes hosted on Artifactory. I put this together with the intent of helping us easily provision our Rancher/Cattle/Docker-based platform wholesale on our machines to test changes before pushing them up.

Here it is: https://github.com/carlosonunez/vagrant_vmware_artifactory_example

Tests are to be added soon! I’m thinking Cucumber integration tests with unit tests on the helper methods and Vagrantfile correctness.

I also tried to emphasize small, isolated and easily readable methods with short call chains and zero side effects.

The pipeline would look roughly like this:

  • Clone repo containing our Terraform configurations, cookbooks and this Vagrantfile
  • Make changes
  • Do unit tests (syntax, linting, coverage, etc)
  • Integrate by spinning up a mock Rancher/Cattle/whatever environment with Vagrant
  • Run integration tests (do lb’s work, are services reachable, etc)
  • Vagrant destroy for teardown
  • Terraform apply to push changes to production

We haven’t gotten this far yet, but this Vagrantfile is a good starting point.

Sleep better with two simple shortcuts.

TL;DR:

Control + ⌘ + Shift + G, and
Home button triple-click.

Why?

Exposing ourselves to bright screens at night while checking our Facebook feed or reddit posts might not be as harmless as it seems. Tons of research, like this and this suggest that viewing things on bright screens right before bed makes our brains think that we’re in daylight longer than we actually are and, consequently, prevent us from falling asleep sooner than we should be. This combined with our early-start culture has been shown to lead to fatigue, decreased concentration and, in some folks, depression.

Additionally, other research has shown that prolonged exposure to artificial light (like those in most offices or our phones) can, over time, damage our eyes’ ability to adjust to incoming light and weaken their sensitivity to it.

I didn’t notice any of this until a Slashdot post introduced me to Flux several years ago. Before using this application, I was usually tired and sore (I rode my bike much more often back then) most of the time, but didn’t think much of it. I went out often back then, and most of the people I came across were just as or more tired than I was, so I thought I was fine.

I would have never thought that simply cutting blue light at night would have improved my sleeping patterns as much as they did. I was honestly surprised and, since then, intrigued about doing everything I could to improve my sleeping habits.

A few months after (happily) using Flux, I saw a developer on our floor who had the oddest setup I’ve seen up until then: a small, vertically-oriented monitor with a completely dark desktop with huge icons and a huge terminal font size. I didn’t ask him much about it, but given how exceptional he was at what he did, I naturally thought: I need to try this.

I wasn’t ready for what happened next. I had absolutely no idea that copying some developer’s setup would completely transform the way that I worked going forward.

Working on dark desktops like the one above (my current working setup) has helped me:

  • Focus better (white text on a dark background is much more readable),
  • Work longer without cutting into my sleep,
  • Utilize smaller real estate much more efficiently (my ideal monitor is a 19″ widescreen), and
  • Realize just how few companies actually support this (Material Design, I’m looking at you!)

Be jealous of my sweet eye-saving setup.

If you’re interested in giving this a try, here are two shortcuts you can set up easily on your Mac and iPhone that’ll make it super easy to toggle between the two:

For Your Mac

  1. Hit the ⌘ and Space key together to open Spotlight, then type in “Keyboard Shortcuts” and press Enter

  2. On the left hand side, click on “Accessibility” to bring up the Accessibility shortcuts on the right side. Find the “Invert Colors” shortcut on the right side, then click on the checkbox to enable it . Afterwards, click twice on the greyed-out key sequence then hit the Control, Shift, ⌘ and G keys together to activate it.

After enabling it, you can easily switch between light and dark mode by hitting:

Control + Shift + ⌘ + G

Note that this will also invert photos and images. If that creeps you out, hit that key sequence again to go back to normal!

For Your iPhone or iPad

You can also enable dark mode on your iPhone! To do so:

  1. Unlock your iPhone, then tap on Settings to open your iPhone’s settings.

  2. Tap on “General,” then on “Accessibility”.

  3. Find the “Invert Colors” option, then tap on the toggle switch to enable it. Afterwards, scroll all the way down to “Accessibility Shortcut,” then tap on it and then on “Invert Colors” to enable the shortcut.

After doing this, you’ll be able to turn on dark mode by triple-clicking your home button!

I hope this helps you as much as it’s helped me!

You’re a better engineer than you think.

I was quite surprised to discover that thousands of people were members of the “Imposter Syndrome” Google+ group within my first month at Google.

I always thought that getting into Google was probably the best social proof of “making it” that an engineer could receive. The interview process is hard, gruelingly technical, relatively unforgiving and riddled with rollercoasters; many incredibly talented Googlers had to go through the process two or more times before getting in for good. (I went through it twice…sort of.) The engineering talent at Google is nearly limitless; many of the world’s most formidable and accomplished computer scientists, sysadmins and software engineers work or worked at Google doing all sorts of things.

So imagine my surprise when literally tons of engineers join a group expressing how they feel as if they aren’t good enough to be at Google or working alongside people with Wikipedia articles written after them. Perhaps it was a big joke that completely went with my head, but given the many, many internal jokes made about not being good enough to be a Googler that I came across (mostly thanks to Memegen), I had my doubts.

I hate checklists.

I can’t help but feel that every other day, I come across a blog post from a programmer or engineer that I’ve never heard of telling me 15 nicely-edited reasons why I’m not worthy of my job. I’ve never used Haskell. I don’t know what git stack does or how to untangle complicated head conflicts from rogue git commit -forces. My .vimrc is really, really plain, and I still don’t know how to write an emacs plugin despite having used it intermittently for the last three years.

Hell, I think if I tell anyone at any conference that I don’t watch Star Trek, don’t play video games and actually love being a Windows engineer (or simply show them my relatively barren Github profile), I’ll be blacklisted by every professional computing community out there.

I can already feel the angry emails coming.

I really hate checklists telling me how to be a “good” engineer. What does “good” mean anyway? Who sets the benchmark? Aside from my manager and peers (who seem to like me, I think?), who’s judging my “goodness?” My gut feeling is that most engineers are much better than they think, and these are my three guiding principles as to why:

Are you learning?

Technology is all about learning new things. If I had to take a guess, I would be scared if anything less than 15 JavaScript frameworks got released last night. What’s last year’s computing messiah usually becomes passé this year (see: virtual machines vs. containers); the state of configuration management is a quintessential example of this.

Are you learning new things? Are you trying new things? If so, then awesome!

Are you challenging yourself?

Finding a groove and sticking with it is a comfortable place to be. However, I believe that sticking with a groove for too long is an easy way to miss things, or, worse, an easy way to think that you don’t need to learn anything new.

In the beginning of my career five years ago, I was really, really good at VBscript. I knew enough to write and maintain behemoth-sized code and where its (many) oddities were. I got so good at it, I thought that learning PowerShell (then Monad) was more of a pain than a benefit. Setting registry keys was (back then) so much more difficult with the StdRegProv provider than with using the Shell object and calling reg.exe.

Had I invested the time in learning Powershell early, I would have probably invested much more time helping build the language or at least collecting cred on Stack Overflow.

If you’re the smartest person in the room, you’re probably in the wrong room.

Are you keeping yourself challenged? Are you working around people who challenge you? Are you taking on increasingly more challenging work? If so, then you’re awesome!

Giving a ****.

Do you care about the quality of your work? Do you document what you’re doing to teach others? If so, then you’re awesome!

Ditch the checklists.

Merriam-Webster defines passion as a a strong feeling of enthusiasm or excitement for something or about doing something. If you’re passionate about what you’re doing and it shows through your work, in my book, even if it pales in comparison to what it should ideally look like, you’re a good engineer in my eyes. So much easier than checklists in my experience.

About Me

Carlos Nunez is a site reliability engineer for Namely, a human capital management and payroll solution made for humans. He loves bikes, brews and all things Windows DevOps and occasionally helps companies plan and execute their technology strategies.

Concurrency is a terrible name.

I was discussing the power of Goroutines a few days ago with a fellow co-worker. Naturally, the topic of “doing things at the same time in fancy ways” came up. In code, this is usually expressed by the async or await keywords depending on your language of choice. I told him that I really liked how Goroutines abstracts much of the grunt work in sharing state across multiple threads. As nicely as he possibly could, he responded with:

You know nothing! Goroutines don’t fork threads!

This sounded ludicrous to me. I (mistakenly) thought that concurrency == parallelism because doing things “concurrently” usually means doing them at the same time simultaneously, i.e. what is typically described as being run in parallel.
Nobody ever says “I made a grilled cheese sandwich in parallel to waiting for x.” So I argued how concurrency is all about multithreading while he argued that concurrency is all about context switching. This small, but friendly, argument invited a few co-workers surrounding us, and much ado about event pumps were made.

After a few minutes of me being proven deeply wrong, one of our nearby coworkers mentioned this tidbit of knowledge:

Concurrency is a terrible name for this.

I couldn’t agree more, and my small post will talk about why.

In computer science, concurrency is the term used to describe the state in which multiple things are done at the same time within the same “thread” of execution. In contrast, parallelism is used to describe the state in which multiple things are done at the same time across multiple “threads” of execution.
The biggest difference between the two is being able to do multiple units of work simultaneously across multiple processors.

“What about multithreading,” you might ask. “I thought that the whole point of doing things across multiple threads was to do multiple things at once!”

Here’s the thing: today’s processors can only do things one instruction at a time. The massive amount of engineering, silicon and transistors that they have are built to execute one instruction at a time really really really quickly and accurately. What gets executed and when is up to the operating system queueing up work for the processor to do. Operating systems deal with this by giving every process (and their threads) a pre-defined amount of time with the processor called a time slice or quantum.

The processor is even processing instructions when the operating system has nothing for it to do; these instructions are called NOOPs in x86 assembly. (Fun fact: whenever you open up Task Manager or Activity Monitor and see the % of CPU being used, what you’re actually looking at is the ratio of instructions being executed to NOOPs.) Process scheduling is quite the loaded topic that I’m almost certain that I’m not doing justice to; if you’re interested in learning more about it, these slides from an operating systems course from UC Davis describe this really well.

Even though operating systems typically schedule work from processes to be done serially on one processor, the programmer
can tell it to divide the work amongst multiple or all processors on the system. So instead of work from this process being done one instruction at a time, it can be done n instructions at a time, where n is the number of processors installed on a system. What’s more is that since most operating systems typically slam the first processor for everything, processes that take advantage of this can typically get more done faster since they are not competing for as time on the main processor. This approach is called symmetric multiprocessing, or SMP, and Windows has supported it since Windows NT and Linux since 2.4. In other words, this is nothing new.

To make matters more confusing, these days, operating systems will often automatically schedule threads across multiple processors automatically if the application uses multiple threads, so for practicality’s sake, concurrent programming == parallel programming.

TL;DR

Concurrency and parallelism aren’t the same, except when they are. Sort of.

About Me

Carlos Nunez is a site reliability engineer for Namely, a human capital management and payroll solution made for humans. He loves bikes, brews and all things Windows DevOps and occasionally helps companies plan and execute their technology strategies.

Doing something boring? Try this one weird trick! Slackers hate it!

I love writing code and building awesome stuff, but there are times where fighting the urge to Reddit for 14 hours feels like this:

When this happens, I break out my secret weapon: The Pomodoro Technique. The basic premise behind this technique is alternating your time between spending several minutes on nothing but working towards a certain goal (let’s call it the hot period) or deliverable and a few minutes on anything that isn’t work (the cold period). 

While you’re working, you should be doing nothing else except the work unless it’s so critical that it can’t wait. Yes, that includes emails, IMs, and phone calls. This is critical, as this (a) trains you to put a completely unfettered focus into something, and (b) makes getting through that tough period a lot faster.

Your cold period, on the other hand, can be spent however way you want as long as it’s only for a few minutes. The cold period should be much, much shorter than the hot period; otherwise, you’ll run the risk of falling off and potentially wasting a lot of time.

My hot period is 30 minutes and my cold period is 10. For getting through a slump, this setup makes it just tolerable enough to get through the hill and the break just short enough to prevent falling into the deep end.

The official technique recommends a desk-side timer (I’m assuming to train your mind into eventually entering hot/cold periods automatically…or something), but I’ve found that any ol’ timer works just fine. I use my iPhone.

This is tom-foolery. There’s no way that this works.

Except it does! And for three reasons:

  1. It gives you something to look forward to after a few minutes of work, even if it’s short,
  2. It helps break down large and seemingly-unending challenges into smaller, more digestible ones, which makes it easier to see what the goal actually is, and
  3. It makes you feel accomplished, which will make you feel more encouraged to continue doing work so you can keep feeling accomplished.

Still not sure?

Try it for a week. Let me know how it goes!

About Me.

I’m the founder of caranna.works, an IT engineering firm in Brooklyn that builds smarter and cost-effective IT solutions that help new and growing companies grow fast. Sign up for your free consultation to find out how.http://caranna.works.

Technical Thursdays: DNS, or why using the Internet is kind of like going to Starbucks

This Thursday, we’ll talk about a system that has been extremely critical (and extremely taken for granted) for shaping the Internet as we know it: the domain name system, or DNS for short.

Before I explain what DNS is, I’ll talk about something I try really hard to hate but ultimately can’t: Starbucks.

I go to Starbucks at least once a day. Given that Google has more coffee machines (and baristas!) sitting idle than my handy downstairs Starbucks does on even their busiest days, this is slightly embarrassing to admit. I love their drinks, but as a recovering coffee snob, I passive-aggressively hate that I love their drinks. My relationship with that Seattle staple is kind of like how a lot of people feel about Taylor Swift: they’ll hate on her forever but will never admit to playing 1989 on repeat.

Wait, that’s just me?

Okay. I can live with that.

Anyway, what I find fascinating about Starbucks aside from their many variants of non-coffee coffee drinks (that are so good but so bad) is how baristas communicate drinks to each other. Somehow, someway, your order for a tall caramel-flavored latte with soy milk, whip cream and a double-shot of espresso is always a tall caramel whip redeye latte to every Starbucks barista on the planet, but trying that on a barista at Cafe Grumpy will usually get you banned for life.

What’s even more fascinating about this is that DNS works “exactly” the same way when you go to BuzzFeed.com on your phone or computer to endlessly browse lists of cat pictures and gifs of people doing funny things.

(Don’t pretend like you don’t.)

You probably know that underneath the the lists and relationship videos, BuzzFeed is really a ton of servers doing lots of hard work to deliver this quality content, and buzzfeed.com is just one of the servers that shows them to you.

What you might not know is that the name of that server isn’t buzzfeed.com; it’s actually: 54.241.35.79. That’s it’s IP address.

If you type in those four (or eight) numbers into Chrome (or whatever your browser of choice is; I use Safari for reasons that won’t be discussed here to avoid an intense holy war), it’ll take you right to BuzzFeed.

How does your computer know that these two things go to the same place? The answer is DNS.

What Is This DNS Magic That You Speak Of?

DNS is a system that maps names like buzzfeed.com or Wikipedia.org to IP addresses. It was created in the early 1980s when the Internet was much much MUCH smaller and has been iterated and improved upon significantly since then. Here’s the original RFC that describes how it works, and surprisingly, a lot of it has held up over time!

These mappings are stored in records. There are several kinds of them. The name-to-IP mapping that I described earlier is stored in an A record, but a DNS can also have records for other mappings to things like shortcuts to A records (CNAME records), mail servers on the network to which that IP address belongs (MX records) or random data (TXT records).

When your computer attempts to find the IP address for a web site, its DNS client (also called a resolver) performs a DNS query. The response it gets back is the DNS response.

So original, I know.

Dots and zones

The dots in a website URL are very important. Every word behind each dot is called a DNS domain, and every one of those words maps to something.

The last word in the URL, i.e. the .com, .org and .football, is called a top-level domain or TLD. Every single one is maintained by the Internet Assigned Numbers Authority, or the IANA. In the early days of simple Internet, this used to give you an idea of what the website was for. .coms were for commercial use or companies, .orgs were for non-profits and foundations, .net were for personal websites and country-specific TLDs like .us or .it were for government-run websites.

However, like most things from that time period, that’s gone completely out the window (do you think bit.ly is in Libya?).

Records within a DNS are broken up into zones, and servers within the DNS are responsible for upholding their zone. These zones are usually HUGE text files that get stored completely within that server’s memory for really fast access. When your computer sends a DNS query, the DNS server you’re configured to use will ask for this server if it doesn’t have the record it’s looking for stored anywhere. It does this by asking for a special record called the State-Of-Authority, or SOA, which tells it where to go next in its search.

DNS is so hot right now

Almost every single web site you’ve visited within the last 20 years or so has likely taken advantage of DNS. If you’re like me, that’s probably a lot of websites! Furthermore, many of the assets on those web sites (think: images and code for all of those fancy site effects) are referred to by name and resolved by DNS.

The Internet as we know it would not function without DNS. As of yesterday, the size of the entire Internet was just over 1 BILLION unique web sites (and growing! exponentially!) and used by over 3 BILLION people.

Now imagine all of that traffic being handled by a single Dell server somewhere in this vast sea of Internet.

You can’t? Good. Me neither.

DNS at WEB SCALE

So how does DNS manage to work for all of these people for all of these web sites? When it comes to matters of scale, the answer is usually: throw a metric crap ton of servers at it.

DNS is no exception.

The Root

There are a few layers of servers involved in your typical DNS query. The first and top-most layer starts at the DNS root servers. These servers are ran by the Internic and are used to tell you which servers own what TLDs (see below).

There are 13 root servers throughout the world, {A through M}.root-servers.net. As you can imagine, they are very, very, very powerful clusters of servers.

The TLD companies

Every TLD is managed by a company. The DNS servers run by these companies contain the records for every website that uses those TLDs. In the case of bit.ly, for example, the records for bit.ly will live on a DNS server managed by the IANA, whereas the records for stupidsiteabout.football will be managed by Donuts.

Whenever you buy a domain with GoDaddy, (a) you are doing yourself a disservice and need to get on Gandi or Hover right now, and (b) your payment gives you the ability to create records that eventually land up on these servers.

The Public Servers

The next layer of servers in the query are the public DNS servers. These are usually hosted by either your ISP, Google or DNS companies like Dyn or OpenDNS, but there are MANY DNS servers available out there. These are almost always the DNS servers that you use on a daily basis.

While they usually have the same set of records that the root servers have, they’ll refer to the root servers above if they’re missing anything. Also, because they are used more frequently than the root servers above, they are often more susceptible to people doing bad things, so the good DNS servers will implement lots of security enhancements to prevent these things from happening. Finally, the really big DNS services usually have MANY more servers available than the root servers, so your query will always be responded to quickly.

Your Dinky Linksys

The third layer of servers involved in the queries most people make aren’t actually servers at all! Your home router most likely runs a small DNS server to help make responses to queries a lot faster. They don’t store a lot of records, and they are typically written pretty badly, so I often reconfigure these routers for my clients so that use Google or OpenDNS instead.

Your job probably has DNS servers of their own to improve performance and also upkeep internal and private records.

Your iPhone

The final layer of a query ends (well, starts) right at your phone or computer. Your computer’s DNS resolver will often store responses to common queries for a short period of time to avoid having to use DNS servers as often as possible.

While this is often a very good thing, this often causes problems when records change. If you’ve ever tried to go onto a website and were unable to, this is often one reason why. Fortunately, fixing this is as simple as clearing your DNS cache. In Windows, you can do this by clicking Start, then typing cmd /c ipconfig /flushdns into your search bar. Use these instructions to do this on your Mac or these instructions to do this on your iPhone or iPad.

This is starting to get long and I’m in the mood for a caramel frap now, so I’m going to stop while I’m ahead here!

Did you learn something today? Did I miss something? Let me know in the comments!

One weird trick that might make your MacBook less janky

A Macbook In Perfect Condition

I was trying to put a bunch of slides together today but had a lot of trouble doing it because my Mac would freeze up every minute or so for about 10-15 seconds. If you’ve ever tried mowing a lawn with no gas, you kind of know how this feels. It was infuriating.

In search of anything that might improve the state of things, I stumbled upon this interesting solution that seems to have made the slowness go away!

If your Mac is freezing up or acting slow in general, give this a try:

  1. Open a Terminal by holding Command (⌘) and Space, typing “Terminal” then hitting Enter.

  2. When the Terminal starts up, type in (or copy and paste): sudo rm /Library/Preferences/com.apple.windowserver.plist. Type in your password when prompted; this is safe.

  3. When that finishes, type in (or copy and paste): rm ~/Library/Preferences/ByHost/com.apple.windowserver*.plist. The terminal might say that there is “no such file or directory;” that is normal (this means that it couldn’t find some files).

  4. When that finishes, shutdown your MacBook then turn it on again but press and hold Command (⌘), Option (⌥), P then R before the Apple logo comes up. This will reset some hardware configuration data, which isn’t critical. (None of your files are affected.) If you did it right, your screen might flicker once. After that happens, press the Power button.

Try it and let me know what you think!

About Me

I’m the founder of caranna.works, an IT engineering firm in Brooklyn, NY that employs time-tested and proven solutions that help companies save lots of money on their IT costs. Sign up for your free consultation to find out how. http://caranna.works.