Veeam Vanguard Summit 2019 Prague – Day 2 Recap

by

Containers and Veeam

David Hill introduces us to the topic of containers, and how Veeam can handle that. Containers make it possible to use less different operating systems. Today you’ve got patches and updates to install on your operating system, no matter if it’s Linux or Windows. With containers, it becomes a lot simpler.

It’s all about buzzwords. Being or getting cloud-ready, being cloud-native, and being it now. A few years ago it was the same with the cloud. Cloud did help to solve some problems, but some problems are still the same, just on a bigger scale. The same for containers. They will help to solve some problems, but some problems will still be problems, just on a different scale.

David explained some things about the statefulness of a container. Containers are good when they are used for what they are thought. But having just any kind of application containerized doesn’t work. When an application fails, and the container management spins up a new instance of that application, the application itself doesn’t know what happened. If you click save in WordPress for example, WordPress doesn’t know what you did before when the application crashed in the backend.

Cloud Tier

Andrew Zhelezko and David Hill are talking about the Cloud Tier. In Veeam, you can set up and define multiple types of backup repositories. You can have local disk storage, NAS, deduplication appliances, etc. You can even combine them in a Scale-out Repository. Now with the latest VBR version you’re even able to scale-out to the cloud with the usage of object storage. You’ve got nearly unlimited storage available in the cloud to store your backups. And in Veeam, that’s all policy-driven. Depending on your policy, Veeam does the automatic cloud-tiering to the object storage of your choice. And you can put that object storage also into a Scale-out Repository as a capacity tier.

That means that you might have some backups local, maybe because of compliance reasons or to meet a certain RPO/RTO time. Everything which is older than a specified time will be tiered out to the object storage.

A new feature is the Immutability of your backup files in the object storage. That means that you can lock the backups which are moved out to the cloud. That gives you protection against rogue admins or to have a certain level of compliance. No one can delete your backup files on the cloud storage. You can set the lock for a specific time, and during this time the backup files can’t be deleted. These settings have to be set on a specific S3 bucket.

Read more

Veeam Vanguard Summit 2019 Prague – Day 1 Recap

by

Welcome Vanguards!

Nikola Pejkova and Rick Vanover welcome all new and renewed Vanguards here at Prague. As we know it from last year, and maybe also from other conferences, there are the famous traffic light signals. In the following days we will see and hear much of green, yellow and red stuff, which means that they are free to publish, embargoed (publish it somewhen) or even under NDA (don’t even think to publish it or talk about it).

Rick and Nikola explained some new things about the Vanguard program to us. Veeam likes to have more engagement with the Vanguards. There will be more recap movies like they did at VeeamON, helping to build up the profile of a Vanguard, not promoting Veeam especially. Rick guides us through the agenda. There are two rooms this year, with even more sessions for all kinds of tastes of technology and interests. For the official Vanguard Dinner on Tuesday we will have our own Veeam shuttles from the hotel to the venue and back. Awesome!

Rick shares some more program updates and priorities. Such an event, and also the Vanguard program is a great opportunity to get more information directly from the source. As a Vanguard, you’re also able to give your feedback directly to the responsible persons, like feedback for beta versions, or the program itself. These persons can push it into the right channels to make it probably happen.

Read more

VeeamON Virtual – The Premier Virtual Conference for Cloud data Management

by

This year, Veeam is hosting a virtual conference for Cloud Data Management. Well, not just a virtual conference, but the Premier Virtual Conference for Cloud Data Management. The VeeamON Virtual Conference on November 20 this year is fully packed with sessions about the Vision & Strategy, and you can learn some Implementation Best Practices. You’ll get also vital insights about Cloud-Powered stuff like Office365 backup, Backup as a Service and Disaster Recovery as a Service, as well as valuable information about Architecture & Design of the backup solution.

And I’m happy to be part of the expert lounge during this virtual event. Feel free to stop by, say hello and ask your questions! All you have to do is to visit the VeeamON Virtual website here, register and join the conference on November 20, 2019.

Upgrade VCSA through CLI Installer

by

My team and I were tasked with a global vSphere upgrade on all of our ESXi hosts, hyper-converged systems and our vCenter. We took enough time to get the inventory, check all the hosts for compatibility and test the various upgrade paths. The upgrade will be rolled out in multiple steps due to personal resources (we’re a small team and currently, it’s summer holiday season) and also to avoid too much downtime. In this blog post, I’d like to share some personal experiences regarding the upgrade of our vCenter. It didn’t work as we’ve planned. But in the end, all worked fine. I’d like also to shoutout a big thank you to my team. You guys rock!

Foreword

Before we dive deeply into the vCenter upgrade process, and what happened, I’d like to explain some steps first to better understand our approach and the upgrade process in general.

One of the milestones is (at the writing of this blog post already “was”) the upgrade of our vCenter. We’re using vCenter for our daily tasks like managing virtual workloads, deployment of new ESXi hosts, etc. But before we could upgrade our vCenter from 6.5 to 6.7, we had to do some host upgrades first. Our hyper-converged infrastructure was running 24/7 without getting much care, like care in the form of firmware upgrades. There was just not enough time to do maintenance tasks like this throughout the last few months or maybe years. Maybe some people also were just afraid of touching these systems, I don’t know for sure. The firmware was old but at least the hypervisor was on a 6.0 version and also in pretty good shape as well.

So we’ve scheduled various maintenance windows, planned the hyper-converged upgrades and made sure that we’ve downloaded everything from the manufacturer we need to succeed. The firmware upgrade went well on all hosts. One host had a full SEL log and that caused some error messages. No real issue at all, but some alerts in vCenter on that cluster we had to get rid of.

The firmware upgrade on one of the hyper-converged cluster took about 18 hours. That was expected, somehow, because the firmware was really old, and did not support higher ESXi versions that 6.0. But everything went well and we had no issues at all, expect the full SEL log which then has been cleared.

After that firmware upgrade, we were able to upgrade the ESXi version on all of the hyper-converged clusters to a 6.5 level. This was needed because of some plugins used to manage these hyper-converged systems. Ok, to let the cat out of the bag, we’re using Cisco HyperFlex and the plugin I’m talking about is that HX plugin. The version for ESXi 6.0 wasn’t supported in vCenter 6.7. That’s the reason we had to upgrade the HyperFlex systems first to ESXi 6.5.

As you know for sure, you can’t manage ESXi hosts later than 6.5 in vCenter 6.5. So we had to do a stop here for the moment, but we were now at least able to upgrade our vCenter. All other hosts were already on 6.0 since they were installed, so no issues upgrading to vCenter 6.7.

Oh, did I already mention that our vCenter doesn’t run on-premises but on a cloud provider? No, it’s not VMC on AWS, but some other IaaS provider. That didn’t make it easier.

But let’s dive into the main topic now, enough of explanation, let’s do the hard work now.

Read more

Expand your logical drive to extend a VMFS datastore

by

Recently, I had to add some hardware to an HPE ProLiant DL380 server. Ok, it wasn’t me because this server runs in a location about 3600 miles away from me. But the engineers on-site completed this task. The engineers added some memory and more disks to the server. My task was to add the newly installed disks to the existing ESXi datastore. This was (still is) a standalone ESXi server, as we say a black box server. It’s a standard HPE ProLiant server with local disks and an SD-Card as ESXi boot disk and it is centrally managed in a vCenter. Local IT persons have limited access to vCenter just to manage their workloads on that specific ESXi black box. There isn’t running much on these black boxes, most of all an SCCM distribution point because lacking enough bandwidth. But anyway, that’s not the topic here.

I want to show you what steps I’ve missed in the first attempt and how I’ve managed to fix it.

As there is not much running on this black box server, it was easy to schedule a maintenance window to shut down the workloads and also the ESXi server, so the engineers onsite were able to install the hardware (memory and disks). Through the iLO interface, I’ve started the server and accessed the Smart Storage Administrator, which is part of the Intelligent Provisioning tool kit on servers of Gen9 and later. It was easy to add the unassigned disk to the already existing RAID array. It took some hours to rebuild all the data because all data had to be redistributed over all disks, with parity and everything needed.

After the server was up and running again, I tried to increase the VMFS datastore capacity. It didn’t work as expected. I didn’t see any device nor LUN which I could extend. That made me curious.

Well then, back to the drawing board…

It wasn’t easy this time to schedule a maintenance window, but I’ve asked the responsible person if he could suggest one. In the meantime, I was digging through the internet to find out what’s wrong or what I’ve missed. I’ve found out that just adding the new disks to the existing RAID doesn’t solve that issue alone. I also had to expand the logical drive. That was the key! So ok, could this be done without another downtime? Thankfully yes!

But before we go deeper here, please, always take a backup of your workloads first. Just in case. Better safe than sorry!

It is indeed possible to expand the logical drive on an HPE ProLiant server without downtime. I’m talking here of a maybe easy, not so complex task. It’s not like I’m going to create new arrays or change the RAID mode. No, just expanding the logical drive.

First, I connected to that ESXi server with SSH to see if the HPE tools were installed. And they were. It’s highly recommended that you use the custom VMware ESXi ISO image to install your server when they come from a vendor like HPE or DELL. These images include all the necessary drivers for your hardware, like network or storage controller, and most of all, they include also some nifty tools as well.

In my case, I’m using the tool “hpssacli”. This tool is just the command line version of the Smart Storage Administrator (HP SSA CLI => Smart Storage Administrator CLI). Nice, isn’t it? 😉

Take me to the CLI, please!

I’ve needed only a few commands to get the things in order. Let’s go into it!

First, I’ve checked the logical drives on the controller:

/opt/hp/hpssacli/bin/hpssacli ctrl slot=0 ld all show status

Mostly, the controller is installed in slot 0. I’m talking here about the P440ar which is on-board when I’m not wrong, so definitely on slot 0. With “ld all” it will display all logical drives configured on that controller.

Next, as I’ve got now the logical drive ID, I’ve checked the details for that logical drive:

/opt/hp/hpssacli/bin/hpssacli ctrl slot=0 ld 1 show

This gave me an overview of that logical drive and I saw that it was just half of the expected size because disks have been added here.

Just to make sure, I’ve checked the controller to see if all disk drives were assigned:

/opt/hp/hpssacli/bin/hpssacli ctrl slot=0  show config

The output showed me that all installed drives were assigned and that there were no unassigned drives. As this controller only had one logical drive, all disk drives were assigned there.

Ok, so then it should be possible to extend the logical drive. And it is, with the following command:

/opt/hp/hpssacli/bin/hpssacli ctrl slot=0 ld 1 modify size=max forced

This command extends the logical drive, so it is going to use the whole disk space as defined in the RAID.

After that, I was able to increase the VMFS datastore capacity through the vSphere web client without problems.