Liveblog: Introduction to Managed Database Services on AWS

This is a liveblog of the AWS re:Invent session titled “Introduction to Managed Database Services on AWS” (DAT307). The speakers for the session are Steve Hunt, Alan Murray, and Robin Spira, all of FanDuel; and Darin Briskman, from AWS Database Services.

Briskman kicks off the session with a quick review of AWS’ managed database offerings. These fall into four categories, which Briskman reviewed so quickly I couldn’t capture. I think they were SQL, NoSQL, data warehousing, and something else. Why use managed databases? Because this allows AWS to take over the responsibility for OS maintenance, DB maintenance, high availability, scalability, etc. All you have to worry about it is the application that runs on the database.

What are the managed relational database services that AWS offers?

• Amazon RDS (Relational Database Service): The oldest service, now supporting MySQL, MariaDB, PostgreSQL, Microsoft SQL Server, and Oracle
• Amazon Aurora: MySQL-compatible (and now PostgreSQL-compatible per the announcement today) with greater scalability, better performance, transparent encryption, high availability, and integration with AWS Lambda

Relational databases are really helpful in many cases, but sometimes NoSQL databases would be more helpful. AWS also offers DynamoDB, which is a managed NoSQL database service. DynamoDB is always clustered, and Continue reading

Liveblog: How News UK Centralized Cloud Governance

This is a liveblog of the AWS re:Invent session titled “How News UK Centralized Cloud Governance Using Policy Management” (DEV306). The speakers for the session are Joe Kinsella from CloudHealth Technologies and Iain Caldwell of News UK/News Corp EMEA.

Kinsella kicks things off by indicating that the session will attempt to tackle the burning question: how does one maintain the agility that brought you to the cloud in the beginning, but enforce the proper level of governance and control? Kinsella and Caldwell then spend a few minutes on introductions before diving into the content of the session.

Caldwell starts off the session content with a review of News Corp’s use of AWS. News UK is currently running 69% of their workloads in the public cloud, with an aim to hit 75% by July 2017. Before they started their journey to the public cloud, News Corp ran a “global application assessment”—and Caldwell believes that this was critical to the success News Corp/News UK has seen so far. News is using a wide variety of AWS services: EC2, S3, VPC, Direct Connect, Route 53, CloudFront, CloudFormation, CloudWatch, RDS, WorkSpaces, Storage Gateway.

When prompted by Kinsella, Caldwell indicates that EC2 instances were the Continue reading

Liveblog: Automating Cloud Mgmt and Deployment

This is a liveblog of the AWS re:Invent session titled “Automating Cloud Management and Deployment for a Diverse Enterprise Application Portfolio” (DEV319). The speakers for the session are David Lowry and Amul Merchant, both from Infor Global.

Merchant kicks the session off with a brief overview of Infor and its cloud strategy. Infor’s CEO, Charles Phillips, was quoted from AWS re:Invent 2014 as having said, “Friends don’t let friends build data centers.” Merchant spends a pretty fair amount of time (too much time, in my opinion) explaining Infor and Infor’s offerings, with only minimal references to how this affects or is affected by the core topic of the presentation. He makes numerous references to “the DevOps toolset” that Infor uses, but does not provide any details or information on said toolset. Instead, the information shared is far too basic for a 300-level session.

After 15 minutes, Lowry takes the stage to talk about the DevOps toolset. The key principles Infor used in building/selecting tools were:

• Automate end-to-end
• Use AWS services wherever possible (this ties the tools closely to AWS)
• Minimally Viable Product (MVP) first, then expand

Some of the tools Infor uses:

• Faro (more information on that in a Continue reading

AWS re:Invent 2016 Keynote with Andy Jassy

This is a liveblog of the Wednesday keynote at AWS re:Invent 2016. Today’s keynote is led by Andy Jassy, CEO of Amazon Web Services. The crowd gathered for the keynote is pretty immense, despite the availability of numerous overflow locations spread across the multiple re:Invent venues.

At precisely 9am, the DJ rocking the pre-keynote music leaves the stage and AWS welcomes Andy Jassy, CEO, to the stage. This is only the 5th re:Invent conference, and Jassy confirms that this year’s attendance is 32,000 with another 50,000 listening in via the live stream.

Jassy starts with an update on the AWS business. As of Q3, AWS is a nearly $13B run-rate business with millions of active customers. Jassy says that nearly every industry segment is using AWS in a “meaningful way,” as is the public sector. He also calls out all the various AWS partners and systems integrators that have built practices on top of AWS, and the “thousands” of ISVs that have built (or rebuilt) products to run on AWS. AWS is, according to some statistics provided by Jassy, the fastest-growing enterprise IT technology company.

In 2014, AWS said the cloud was the “new normal.” In 2015, AWS said Continue reading

Thoughts on AWS re:Invent Day 1

As I wrap up Day 1 of AWS re:Invent 2016 in Las Vegas (can I consider today to be day 1?), I wanted to capture a quick summary of thoughts about the sessions, the content, the attendees, and the event (not necessarily in that order).

First, here are some links to the liveblogs I posted from today’s sessions:

Liveblog: Scaling to Your First 10 Million Users
Liveblog: Hybrid Architectures, Bridging the Gap to the Cloud
Liveblog: Getting the Most Bang for Your Buck with EC2

Overall, the sessions have been pretty decent so far. Some portions of some of the sessions feel more like a sales pitch than an educational session, but I’m sure that’s the case at other events as well (yes, I’m talking about VMworld). I’m not yet sure if the nature of what AWS does/offers lends itself to subjectively feel more like a sales pitch or not. Case in point: how does a presenter suggest to attendees—for solid technical reasons—that they should consider using a service like Route 53 or DynamoDB or SQS (or any one of a dozen other services) without it also sounding like a sales pitch?

From an attendee perspective, I’ve been “badge Continue reading

Liveblog: Getting the Most Bang for Your Buck with EC2

This is a liveblog of the AWS re:Invent session titled “Getting the Most Bang for Your Buck With #EC2 #Winning” (CMP202). The speaker for the session is Joshua Bergin, General Manager, EC2 Spot Business. According to the abstract, this session is supposed to focus on effectively using on-demand instances versus spot instances and reserved instances.

As a matter of quick introduction, there are three purchasing options for EC2:

• On-demand: “pay as you go”; no long-term commitments
• Reserved: good for steady-state workloads, used with 1 yr or 3 yr commitment
• Spot: pay market price for unused compute capacity

How do you choose which one to use? Bergin shares the “four pillars of performance and cost optimization”:

1. Right-sizing: choosing the cheapest instance available while meeting performance requirements
2. Purchasing options: Bergin will discuss this in more detail; this is the primary focus of the discussion
3. Increase elasticity: turning off (“scaling down”) instances that don’t need to be running (example: turn off development workloads when the developers aren’t working)
4. Measure, monitor, and improve: tagging resources; identitying always-on instances; identifying instances that can be downsized; recommending Reserved Instances (RIs) where it makes sense; dashboards and reports

Bergin points out the key AWS pricing principles (no Continue reading

Liveblog: Scaling to Your First 10 Million Users

This is a liveblog of the AWS re:Invent session titled “Scaling to Your First 10 Million Users.” It’s my first session of the week here at re:Invent; yesterday’s sessions were full and I couldn’t get into anything. (The crowds here at the event are pretty significant; I think I heard 32K attendees total.) The speaker for the session is Joel Williams, an AWS Solutions Architect.

Williams starts out with a brief blurb about how this session is a perennial favorite at re:Invent, and how the principles are fundamental to working in building solutions in/on AWS. Even if attendees don’t have the sort of immediate scaling needs that Williams may be describing in this session, he believes that the lessons/fundamentals he discusses are applicable to lots of customers, lots of applications, and lots of use cases.

Williams starts out by saying that while Auto-Scaling is a destination on customers’ scaling journey, it’s not where you want to start. It’s not a “magic button” that fixes all problems. Williams puts up a map that shows AWS’ 14 global regions, encompassing 38 different availability zones, and points out that availability zones are a fundamental building block for highly-available applications. The next Continue reading

Liveblog: Hybrid Architectures, Bridging the Gap to the Cloud

This is a liveblog of the AWS re:Invent session titled “Hybrid Architectures: Bridging the Gap to the Cloud” (ARC208). The line to get into this session, as with the previous session, was quite long—and that was for attendees who’d already registered for the session. Feedback I’ve heard from folks who weren’t registered for sessions was that they weren’t getting in, period. The speaker for the session is Jamie Butler, Manager of Solutions Architecture at AWS (focused on state/local government).

Butler starts out by establishing some expectations—attendees should be familiar with regions, AZs (this is a 200-level talk), and will focus on hybrid use cases. Butler says there will be some demos along the way. This session will not focus on the VMware announcement regarding VMware Cloud on AWS.

Butler then quotes Werner Vogels in saying that adopting cloud is not an all-or-nothing proposition. With that in mind, Butler transitions into a discussion of a particular customer example. In this case, the customer had Active Directory, a file server, and a bunch of Windows-based desktops connecting back to the file server for data access.

The first thing to tackle in a scenario like this is identity. Butler says you don’t want Continue reading

Installing Ansible 2.2 on Fedora 25

As part of my ongoing investigation of the usability of various Linux distributions and desktop environments, I’ve been working with Fedora 25. As part of the investigation I need to see how to perform certain tasks, one of which is working with Ansible. As a result, I needed to install Ansible 2.2 on Fedora 25, and it turns out it wasn’t as simple as pip install ansible.

I generally prefer to run Ansible in a Python virtualenv, but I don’t believe that it will make any difference to this procedure. However, I’m happy to be corrected if someone knows otherwise.

To create a Python virtualenv, you’ll first need virtualenv installed. I prefer to install virtualenv globally for all users using this command:

sudo -H pip install virtualenv

Once virtualenv is installed, then create a virtualenv for Ansible:

virtualenv ~/Envs ansible

Then activate the virtualenv:

source ~/Envs/ansible/bin/activate

At this point, you can try a pip install ansible, but it will fail. First, you need to install some additional development libraries that are required in order to install Ansible:

sudo dnf install libffi-devel redhat-rpm-config python-devel openssl-devel

Once those packages are installed, then you’re finally ready to install Ansible into Continue reading

Using GNOME Keyring for Git Credentials on Fedora 25

In this post, I’m going to show you how to use the GNOME Keyring on Fedora 25 as a credential helper for Git. This post is very closely related to my earlier post on using GNOME Keyring as a Git credential helper on Ubuntu 16.04. As with the earlier Ubuntu-related post, what I’m including here isn’t new or ground-breaking information; I’m posting it primarily to make the information easier to find for others.

Like Ubuntu 16.04, Fedora 25 already has the basis for integrating GNOME Keyring into Git as a credential helper already installed into the /usr/share/doc/git-core-doc/contrib/credential/gnome-keyring directory.

Unlike Ubuntu 16.04, though, Fedora already has a compiled credential helper installed. This Git credential helper is found at /usr/libexec/git-core/git-credential-gnome-keyring. This credential helper is ready to use.

To get GNOME Keyring support for storing Git credentials, then, all one has to do is simply configure Git appropriately (no need to install additional packages or compile anything). You can configure Git via a couple of different ways:

1. You can use the git config command, like this:

    git config --global credential.helper /usr/libexec/git-core/git-credential-gnome-keyring
   

2. You can edit ~/.gitconfig directly, using the text editor of your choice. Add this text:

    [credential]
    helper  Continue reading

Using GNOME Keyring as Git Credential Helper

In this post, I’m going to show you how to use the GNOME Keyring on Ubuntu 16.04 as a credential helper for Git. This post stems from my work in transitioning to Linux as my primary OS, an effort I’ve ratcheted up significantly in the last few weeks. What I’m including here isn’t new or ground-breaking information; I’m posting it primarily to make the information easier to find for others.

On Ubuntu 16.04, the basis for integrating GNOME Keyring into Git as a credential helper is already installed into the /usr/share/doc/git/contrib/credential/gnome-keyring directory. However, if you try to simply run sudo make in that directory, it will fail. In order to make it work, you must first install some additional development libraries:

sudo apt install libgnome-keyring-dev

Once you’ve installed this additional package, running sudo make in that directory will quickly compile a binary named git-credential-gnome-keyring. Once you have that binary, then you can configure Git to use GNOME Keyring as a credential helper. You can do this a couple of different ways:

1. You can use the git config command, like this:

    git config --global credential.helper /usr/share/doc/git/contrib/credential/gnome-keyring/git-credential-gnome-keyring
   

2. You can edit ~/.gitconfig directly, using the text editor of your Continue reading

Spousetivities at AWS re:Invent

For the first time ever, Spousetivities will be at AWS re:Invent in Las Vegas! This means that IT pros traveling to Vegas can plan on bringing along their spouse, partner, significant other, or family member(s) and know that there are fun, safe, organized activities for them.

So what’s planned? Here’s a quick look:

• More cooking fun with Chef Phillip Dell, a previous season winner on Food Network’s “Chopped” show
• Hoover Dam and Grand Canyon tour (a must-see if you haven’t been before)
• Vegas food tour (hosted by Chef Dell, of course!)

As is a tradition with Spousetivities, participants in the activities have the chance to win prizes like iPads, Bose headphones, VR headsets, and lots of other goodies.

Crystal’s blog post on the Spousetivities site has a few more details, and registration is open right now.

Technology Short Take #73

Welcome to Technology Short Take #73. Sorry for the long delay since the last Technology Short Take; personal matters have been taking quite the toll (if you follow me on Twitter, you’ll know to what personal matters I’m referring). In any case, enough of that—here’s some data center-related content that I hope you find useful!

Networking

• Ansible has made some good progress in supporting network automation in the latest release (2.2), according to this blog post. This is an area where I hope to spend more time in the coming weeks before years’ end.
• Tomas Fojta shows how to use a PowerShell script to monitor the health of NSX Edge gateways.
• Jeremy Stretch mulls over the (perceived) problem of getting traffic into and out of overlay networks. I recommend reading this article, as well as reading the comments. Many commenters suggest just using L3 and having the hosts participate in a routing protocol like BGP, but as Jeremy points out many switches don’t have the capacity to handle that many routes. (Or, if they do, they’re quite expensive.) Seems like there’s this company in Palo Alto making a product that handles this issue pretty decently…(hint).
• Cumulus Continue reading

An Introduction to the VirtualBox CLI

This post provides a basic introduction to the VirtualBox CLI (command-line interface) tool, vboxmanage. This post does not attempt to replace the comprehensive documentation; rather, its purpose is to help users who are new to vboxmanage (such as myself, having recently adopted VirtualBox for my Vagrant environments) get somewhat up to speed as quickly and as painlessly as possible.

Basic Commands

Let’s start with some basic operations. Here are a few to get you started:

• To list all the registered VMs, simply run vboxmanage list vms. Note that if you are using Vagrant with VirtualBox, this command will also show VirtualBox VMs that have been instantiated by Vagrant. Similarly, if you are using Docker Machine with VirtualBox, this command will show you VMs created by Docker Machine.

• To list all the running VMs, use vboxmanage list runningvms.

• To start a VM, run vboxmanage startvm <name or UUID>. You can optionally specify a --type parameter to control how the VM is started. Using --type gui will show it via the host GUI; using --type headless means you’ll need to interact over the network (typically via SSH). To emulate Vagrant/Docker Machine-like behavior, you’d use --type headless.

• Once a VM is Continue reading

Thinking Out Loud: The Future of Kubernetes

I’ve just wrapped up KubeCon/CloudNativeCon 2016 in Seattle, WA. There’s no doubt the Kubernetes community is active and engaged, and the project itself is charging forward. As both the community and the project grow, though, what does that mean for the future of Kubernetes?

Here are my thoughts, hopefully presented in a somewhat logical fashion.

It seems to me that Kubernetes has been successful thus far because of a strong focus on the problem it’s trying to solve. You can see this in the Kubernetes web site, where phrases like “Production-Grade Container Orchestration” and “Automated container deployment, scaling, and management” are found. You can see this in the API abstractions Kubernetes uses (a pod as a group of co-located containers, a service as a stable access point for sets of pods, etc.). You can see it in the real-world customer deployments and use cases. Kubernetes seems focused on addressing the needs of container-based microservices-centric application architectures.

However, there now seem to be some efforts to push Kubernetes to support other types of applications as well. One could look at DaemonSets (which are used to ensure that a particular pod is always running on every node; useful for “infrastructure” services Continue reading

Managing AWS Infrastructure with Ansible

In this post, I’m going to discuss some concepts behind managing your Amazon Web Services (AWS) infrastructure using Ansible. Ansible is a very popular tool for configuring operating system instances and software; using the concepts and examples provided in this post would allow you to expand your use of Ansible to include—when using AWS—the creation and deletion of the operating system instances themselves, as well as related infrastructure components (like security groups or other services).

Preface

Before I continue, I’d like to first discuss the “fit” of using Ansible for this particular purpose. Ansible doesn’t store the state of managed systems. Perhaps this is due to the agentless architecture; I don’t know. What that means in this particular use case is that you must take other steps to store information you’ll absolutely need like instance IDs, security group IDs, and the like because Ansible itself doesn’t. In my mind, this makes Ansible a less-than-ideal tool for this particular use case. That doesn’t mean Ansible isn’t a good tool; it just means that Ansible may not be the best tool for this particular purpose. (Think of it like this: Yes, you can sometimes unscrew something using a knife, but a screwdriver Continue reading

Vagrant-Photon OS Bug and Workaround

I recently came across a bug in using VMware Photon OS with Vagrant, and so in this post I’m going to point out this bug and provide a workaround. The bug is, fortunately, pretty innocuous, and only affects Vagrant environments that configure additional network interfaces to Photon OS VMs. The workaround is equally easy, thankfully.

First, I’ll point out that the fix for this bug has already been pushed to Vagrant, but it hasn’t yet (as of this writing) made it into a release. Vagrant 1.8.6 was the latest release of this writing, and it still exhibits the bug.

There are a number of somewhat-interrelated issues:

1. First, the “vagrant-guests-photon” Vagrant plugin (latest version is 1.0.4) is no longer needed. This code has been replaced by code that is distributed as part of Vagrant itself. This wouldn’t normally be an issue, except that…

2. The plugin relies on awk, which is no longer included in recent releases of the Photon OS Vagrant box. I can’t tell you exactly when this started, but I can confirm the last couple of releases (1.2.0 and 1.2.1) are definitely affected.

3. Finally, the code which replaces the Continue reading

VMworld EMEA 2016 Day 2 Keynote

This is a liveblog of the day 2 general session at VMworld EMEA 2016 in Barcelona, Spain. I wasn’t able to write a liveblog of the day 1 session due to some scheduling/logistical conflicts, but managed to get things arranged for day 2 (well, most of it—I’ll have to cut this short so I can get to a customer meeting).

At 9am, Sanjay Poonen takes the stage to kick off the general session. Poonen walks through a number of examples how “digital transformation” is affecting businesses and organizations across a variety of industry verticals. Poonen positions Workplace One as the “Switzerland” solution that bridges different kinds of applications (Windows client-server apps, web apps, and mobile apps) with different kinds of devices (Apple, Google, Samsung, Microsoft). The key ingredients of Workspace One are VDI, EMM, and identity.

Poonen quickly transitions into a demo of Workspace One on an iPhone, showing off how VMware employees use Workspace One to run apps like Workday, Concur, ADP, Boxer (VMware’s mobile e-mail client), AirWatch Content Locker, and others. The demo then moves into a demonstration of VDI, including 3-D accelerated graphics, on a Samsung Android tablet. Following the demo, Poonen kicks off a customer testimonial Continue reading

Technology Short Take #72

Welcome to Technology Short Take #72. Normally, I try to publish these on Fridays, but some personal travel prevented that this time around so I’m publishing on a Monday instead. Enough of that, though…bring on the content! As usual, here’s my random collection of links, articles, and thoughts about various data center technologies.

Networking

• Charles Min-Cheng Chan has a write-up on using IPv6 in Mininet.
• How’s this for Technology Short Take inception—in TST #70, I mentioned the Cumulus topology converter. That prompted Cody Bunch to write this article on setting up a topology with BGP on Linux using the topology converter, which I’m now mentioning in this Technology Short Take. (Does your head hurt yet?)
• PowerShell on Cumulus Linux on a network switch? Pete Lumbis shows you how. (There are some very interesting implications around this combination, but I’ll reserve that for a future discussion.)
• Someone pointed me toward this article on using Sophos UTM for remote access to their home lab.
• Lindsay Hill recently took Simon Wardley’s pioneers/settlers/town planners (PST) model—which is itself based on Cringely’s Accidental Empires book and the commandos/infantry/police model—and applied to it the networking industry. It’s a really good read and Continue reading

A Triple-Provider Vagrant Environment

In this post, I’d like to share with you some techniques I used to build a triple-provider Vagrant environment—that is, a Vagrant environment that will work unmodified with multiple backend providers. In this case, it will work (mostly) unmodified with AWS, VirtualBox, and the VMware provider (tested with Fusion, but should work with Workstation as well). I know this may not seem like a big deal, but it marks something of a milestone for me.

Since I first started using Vagrant a couple of years ago, I’ve—as expected—gotten better and better at leveraging this tool in a flexible way. You can see this in the evolution of the Vagrant environments found in my GitHub “learning-tools” repository, where I went from hard-coded data values to pulling data from external YAML files.

One thing I’d been shooting for was a Vagrantfile that would work with multiple backend providers without any modifications, and tonight I managed to build an environment that works with AWS, VirtualBox, and VMware Fusion. There are still a couple of hard-coded values, but the vast majority of information is pulled from an external YAML file.

Let’s take a look at the Vagrantfile that I created. Here’s Continue reading