VCSA

Backup and Restore vCenter Server Appliance

by

Just a few weeks ago, vSphere 7 saw the light of day. And people went crazy! New ESXi servers with vSphere 7 have sprung up like mushrooms. So many people directly upgraded their homelabs, or maybe even their production systems.

This blog post, I know the last post is some time ago, will show you how you can backup your vCenter Server Appliance with their integrated backup functionality, and also how you can restore it, in case something went wrong. Except for two ways, I went through all options for backup targets and tried to find out how to configure it. So there should be at least one way how you can back up your vCenter data to a proper location in your data center.

Why is it a good idea to back up your vCenter

vCenter is your management central in terms of virtualization. You manage all your ESXi server with it, your clusters, your data center networking maybe (with NSX), you’ve got some automation running, got your host profiles, storage policies, etc. in place there. Why lose all the stuff you’ve configured over a longer period, with maybe much tinkering, try and error? Backing up vCenter is not so hard. You need a backup target, a user and a password. In vCenter 6.7 you can even schedule the backup, which makes things even easier than before, where it wasn’t possible to configure a schedule.

Supported protocols for backup

vCenter supports the following protocols for backup:

  • FTP
  • NFS
  • SMB
  • FTPS
  • SCP
  • HTTP
  • HTTPS

This guide will show you how to configure all of the above protocols, except HTTP and HTTPS. I didn’t see a sense in setting up such a configuration in my lab, because it doesn’t seem to me as such backup targets would exist in a company either. I also think that these two protocols might be the slowest, compared to all the other protocols available. In data centers, no matter if on-premises or cloud, the often-used protocols are NFS and SMB. So chances are high that there might be already a suitable backup target for vCenter. Or it can be easily created. Also, FTP is still commonly used, and we’ve got also secure options with FTPS and SCP.

Backup performance

To be honest, the backup performance was not my top priority. I wanted to configure and test all supported protocols except HTTP and HTTPS. It’s clear that performance matters, at least to a certain degree. Backup windows might be small, or systems should not be impacted with a heavy load. Before we move on, I’d like to show you how the performance was during my tests.

I’ve set up a new vCenter server appliance for this backup and restore test. It is a tiny deployment with 2 CPU, 10GB memory and default disk size (thin). There is nothing configured within vCenter, no hosts, no clusters, nothing except backup. You can see that the amount of data transferred is the same in all tests. In regards to the duration, we’ve got the SMB protocol on the first place, followed by FTPS on the second, and NFS on the third place. Yes, I’m aware of “but there’s ftp:// and not ftps://”. I’ve configured FTPS as you can see later on the screenshots, but when I executed the backup job, it was logged as “ftp”. You can spot the difference at the port used for FTPS.

Read more

Upgrade VCSA through CLI Installer

by

My team and I were tasked with a global vSphere upgrade on all of our ESXi hosts, hyper-converged systems and our vCenter. We took enough time to get the inventory, check all the hosts for compatibility and test the various upgrade paths. The upgrade will be rolled out in multiple steps due to personal resources (we’re a small team and currently, it’s summer holiday season) and also to avoid too much downtime. In this blog post, I’d like to share some personal experiences regarding the upgrade of our vCenter. It didn’t work as we’ve planned. But in the end, all worked fine. I’d like also to shoutout a big thank you to my team. You guys rock!

Foreword

Before we dive deeply into the vCenter upgrade process, and what happened, I’d like to explain some steps first to better understand our approach and the upgrade process in general.

One of the milestones is (at the writing of this blog post already “was”) the upgrade of our vCenter. We’re using vCenter for our daily tasks like managing virtual workloads, deployment of new ESXi hosts, etc. But before we could upgrade our vCenter from 6.5 to 6.7, we had to do some host upgrades first. Our hyper-converged infrastructure was running 24/7 without getting much care, like care in the form of firmware upgrades. There was just not enough time to do maintenance tasks like this throughout the last few months or maybe years. Maybe some people also were just afraid of touching these systems, I don’t know for sure. The firmware was old but at least the hypervisor was on a 6.0 version and also in pretty good shape as well.

So we’ve scheduled various maintenance windows, planned the hyper-converged upgrades and made sure that we’ve downloaded everything from the manufacturer we need to succeed. The firmware upgrade went well on all hosts. One host had a full SEL log and that caused some error messages. No real issue at all, but some alerts in vCenter on that cluster we had to get rid of.

The firmware upgrade on one of the hyper-converged cluster took about 18 hours. That was expected, somehow, because the firmware was really old, and did not support higher ESXi versions that 6.0. But everything went well and we had no issues at all, expect the full SEL log which then has been cleared.

After that firmware upgrade, we were able to upgrade the ESXi version on all of the hyper-converged clusters to a 6.5 level. This was needed because of some plugins used to manage these hyper-converged systems. Ok, to let the cat out of the bag, we’re using Cisco HyperFlex and the plugin I’m talking about is that HX plugin. The version for ESXi 6.0 wasn’t supported in vCenter 6.7. That’s the reason we had to upgrade the HyperFlex systems first to ESXi 6.5.

As you know for sure, you can’t manage ESXi hosts later than 6.5 in vCenter 6.5. So we had to do a stop here for the moment, but we were now at least able to upgrade our vCenter. All other hosts were already on 6.0 since they were installed, so no issues upgrading to vCenter 6.7.

Oh, did I already mention that our vCenter doesn’t run on-premises but on a cloud provider? No, it’s not VMC on AWS, but some other IaaS provider. That didn’t make it easier.

But let’s dive into the main topic now, enough of explanation, let’s do the hard work now.

Read more

Enable Flash Player in Google Chrome 69

by

Some weeks ago Google released the newest version of their Chrome browser, version 69. I thought yeah, get it and update it. I really like Google Chrome because it supports all the websites I’m visiting often (or at least did, but more on that later) and it’s fast. You can also customize it with plugins like mouse gestures or so to customize it for your needs. What I didn’t know is that Google switched off the support for Flash Player as far as I knew it from the older versions. You aren’t able anymore to add websites to the allow or deny list in the Flash settings within the browser. Or at least you can’t add the websites directly on that list in the settings. There are some more clicks to do. Officially announced was the end of Flash Player by Adobe. By the year 2020, Flash Player won’t exist anymore, won’t be supported by Adobe nor by the most used browser software. But there’s a “but”. You can manually add specific websites to the allow or block list of Google Chrome, but not in the way you might know. But why the heck should I use Flash anyway? All my favorite websites are already HTML5 compatible and all stuff works without that crappy Flash plugin! But wait! Do you use the VMware vCenter browser client? Probably the Flex Client because you still have the need for it, like vSAN, Update Manager, or 3rd party plugins of different software and hardware vendors within vCenter? Then you’ll have the same issues as I had. The vCenter Flex Client (aka Flash client) obviously won’t work anymore without Flash. Yes, I know, you don’t need to use the Flex Client for vSAN or the Update Manager because in vCenter 6.7 it is finally available. But what about the 3rd party plugins? There is a lot of stuff out there you probably need, I don’t know your infrastructure. I can only compare with mine. But I can tell you, enabling Flash Player in Google Chrome, even in the most recenter version 69 is easier than you think. It’s just some steps and clicks, no rocket science!

Read more

Recap of the latest VMware vSphere 6.7 releases

by
vSphere 6.7

Oh boy, what a week! Some say that winter is now finally gone, nice and warm weather, not wearing winter jackets anymore. But hey, i’m not a weatherman. When you’re sitting in the office i think it doesn’t matter if it’s raining or snowing outside. Just kidding… Let’s get back to business.

There was some rumor about the next upcoming version. Will it be version 7? Or something just above 6.5? VMware did release several new products versions! And it’s all with version number 6.7. What a list! It’s one of those email notifications that I usually like to scroll down, a little more, and more and more, to get all the news soaked up like a sponge. I’d like to dive in right now and provide you a recap of this weeks VMware releases. And as i said, it’s quite a list. I’ll pick out just some new key features. You can find the full release news on the VMware Blogs (links provided here).

New product versions

vSphere 6.7

  • several new APIs that improve the efficiency and experience to deploy vCenter, to deploy multiple vCenters based on a template, to make management of vCenter Server Appliance significantly easier, as well as for backup and restore
  • significantly simplifies the vCenter Server topology through vCenter with embedded platform services controller in enhanced linked mode
  • 2X faster performance in vCenter operations per second
  • 3X reduction in memory usage
  • 3X faster DRS-related operations (e.g. power-on virtual machine)
  • vSphere 6.7 improves efficiency when updating ESXi hosts, significantly reducing maintenance time by eliminating one of two reboots normally required for major version upgrades (Single Reboot). In addition to that, vSphere Quick Boot is a new innovation that restarts the ESXi hypervisor without rebooting the physical host, skipping time-consuming hardware initialization
  • The HTML5-based vSphere Client provides a modern user interface experience that is both responsive and easy to use, and it’s now including other key functionality like managing NSX, vSAN, VUM as well as third-party components.
  • enabling encrypted vMotion across different vCenter instances
  • enhancements to Nvidia GRID vGPU
  • vSphere 6.7 introduces vCenter Server Hybrid Linked Mode, which makes it easy and simple for customers to have unified visibility and manageability across an on-premises vSphere environment running on one version and a vSphere-based public cloud environment, such as VMware Cloud on AWS, running on a different version of vSphere.
  • vSphere 6.7 also introduces Cross-Cloud Cold and Hot Migration
  • Delivers a new capability that is key for the hybrid cloud, called Per-VM EVC

More information here: Introducing VMware vSphere 6.7 / VMware Blogs

vSAN 6.7

  • vSAN 6.7 provides intuitive operations that align with other VMware products from a UI and workflow perspective to provide a “one team, one tool” experience
  • Iintroduces a new HTML5 UI based on the “Clarity” framework as seen in other VMware products (All products in the VMware portfolio are moving toward this UI framework)
  • A new feature known as “vRealize Operations within vCenter” provides an easy way for customers to see vRealize intelligence directly in the vSphere Client
  • vSAN 6.7 now expands the flexibility of the vSAN iSCSI service to support Windows Server Failover Clusters (WSFC)
  • vSAN 6.7 introduces an all-new Adaptive Resync feature to ensure a fair-share of resources are available for VM I/Os and Resync I/Os during dynamic changes in load on the system
  • Optimizes the de-staging mechanism, resulting in data that “drains” more quickly from the write buffer to the capacity tier.  The ability to de-stage this data quickly allows the cache tier to accept new I/O, which reduces or eliminates periods of congestion
  • New health checks include:
    • Maintenance mode verification ensures proper decommission state
    • Consistent configuration verification for advanced settings
    • vSAN and vMotion network connectivity checks improved
    • Improved vSAN Health service installation check
    • Improved physical disk health check combines multiple checks (software, physical, metadata) into a single notification
    • Firmware check is independent from driver check

More information here: What’s New with VMware vSAN 6.7 / VMware Blogs and also here: Extending Hybrid Cloud Leadership with vSAN 6.7

vCenter Server 6.7

  • The vSphere Client (HTML5) is full of new workflows and closer to feature parity
  • built-in file-based vCenter Server backup now includes a scheduler

Installation

  • No load balancer required for high availability and fully supports native vCenter Server High Availability.
  • SSO Site boundary removal provides flexibility of placement.
  • Supports vSphere scale maximums.
  • Allows for 15 deployments in a vSphere Single Sign-On Domain.
  • Reduces the number of nodes to manage and maintain.

Migration

  • vSphere 6.7 is also the last release to include vCenter Server for Windows, which has been deprecated.
  • migrate to the vCenter Server Appliance with the built-in Migration Tool
  • Deploy & import all data
  • Deploy & import data in the background
  • Customers will also get an estimated time of how long each option will take when migrating

Upgrading

  • vSphere 6.7. will support upgrades and migrations only from vSphere 6.0 or 6.5
  • vSphere 5.5 does not have a direct upgrade path to vSphere 6.7
  • Upgrade path: vSphere 5.5 to vSphere 6.0 or 6.5, and then to vSphere 6.7
  • vCenter Server 6.0 or 6.5 managing ESXi 5.5 hosts cannot be upgraded or migrated until the hosts have been upgraded to at least ESXi 6.0
  • Reminder: end of general support for vSphere 5.5 is September 19, 2018.

Monitoring and Management

  • vSphere Appliance Management Interface (VAMI) on port 5480 has received an update to the Clarity UI
  • There is now a tab dedicated to monitoring. Here you can see CPU, memory, network, database and disk utilization.
  • Another new tab called Services is also within the VAMI, giving the option to start, stop, and restart vCenter Server services if needed
  • vSphere 6.7 also marks the final release of the vSphere Web Client (Flash). Some of the newer workflows in the updated vSphere HTML5 Client release include:
    • vSphere Update Manager
    • Content Library
    • vSAN
    • Storage Policies
    • Host Profiles
    • vDS Topology Diagram
    • Licensing

More information here: Introducing vCenter Server 6.7 / VMware Blogs

vSphere with Operations Management 6.7

  • new plugin for the vSphere Client. This plugin is available out-of-the-box and provides some great new functionality
  • When interacting with this plugin, you will be greeted with 6 vRealize Operations Manager (vROps) dashboards directly in the vSphere client
  • overview, cluster view, and alerts for both vCenter and vSAN views
  • The new Quick Start page is making it easier to get directly to the data you need to
  • four use cases: Optimize Performance, Optimize Capacity, Troubleshoot, and Manage Configuration
  • The Workload Optimization dashboard was updated. Workload Optimization takes predictive analytics and uses them in conjunction with vSphere Distributed Resource Scheduler (DRS) to move workloads between clusters. New with vROps 6.7, you can now fine tune the configuration for workload optimization
  • vROps 6.7 introduced a completely new capacity engine that is smarter and much faster

More information here: vSphere with Operations Management 6.7 / VMware Blogs

vSphere 6.7 Security

  • TPM 2.0 support for ESXi
  • Virtual TPM 2.0 for VMs
  • Support for Microsoft Virtualization Based Security
  • UI updates (combined all encryption functions (VM Encryption, vMotion Encryption) into one panel in VM Options)
  • Multiple SYSLOG targets
  • FIPS 140-2 validated cryptographic modules – by default!

More information here: vSphere 6.7 Security / VMware Blogs

Developer and Automation Interfaces for vSphere 6.7

  • Added functionality to existing APIs in vSphere 6.7
  • Coverage of new areas
  • Appliance API updates: from prechecks to staging to installation and validation, it’s all available by API now
  • vCenter API updates: new APIs have been added to interact with the VM’s guest operating system (OS), viewing Storage Policy Based Management (SPBM) policies, and managing vCenter server services
  • also a handful of new APIs to handle the deployment and lifecycle of the vCenter server
  • a handful of updates to the vSphere Web Services (SOAP) APIs as well

More information here: Developer and Automation Interfaces for vSphere 6.7 / VMware Blogs

Faster Lifecycle Management Operations in VMware vSphere 6.7

  • brand-new Update Manager interface which is now part of the HTML5 Client
  • Update Manager in vSphere 6.7 keeps VMware ESXi 6.0 to 6.7 hosts reliable and secure
  • the new UI provides a much more streamlined remediation process, requiring just a few clicks to begin the procedure. It’s not just a port from the old Flash client
  • Hosts that are currently on ESXi 6.5 will be upgraded to 6.7 significantly faster than ever before
  • Several optimizations have been made for that upgrade path, including eliminating one of two reboots traditionally required for a host upgrade
  • Quick Boot eliminates the time-consuming hardware initialization phase by shutting down ESXi in an orderly manner and then immediately re-starting it

More information here: Faster Lifecycle Management Operations in VMware vSphere 6.7 / VMware Blogs

vSphere 6.7 for Enterprise Applications

  • include support for Persistent Memory (PMEM) and enhanced support for Remote Directory Memory Access (RDMA)
  • PMEM is a new layer called Non-Volatile Memory (NVM) and sits between NAND flash and DRAM, providing faster performance relative to NAND flash but also providing the non-volatility not typically found in traditional memory offerings
  • new protocol support for Remote Direct memory Access (RDMA) over Converged Ethernet, or RoCE (pronounced “rocky”) v2, a new software Fiber Channel over Ethernet (FCoE) adapter, and iSCSI Extension for RDMA (iSER)

More information here: vSphere 6.7 for Enterprise Applications / VMware Blogs

VMware – Read before upgrade to vSphere 6.5

by
VMware

Yesterday VMware announced the general availability of the brand new vSphere 6.5. They announced the new version at this years VMworld in Barcelona. But now you can download and install the bits. But there is a catch. Please make sure you read and understand all the important information before upgrading to vSphere 6.5 because there might be some limitation at the moment. Let me bring some light into the darkness.

Compatibility considerations

You should not upgrade to vSphere 6.5 if you are running one (or some / all) of these software components in your environment:

  • VMware NSX
  • VMware Integrated OpenStack
  • vCloud Director for Service Providers
  • vRealize Infrastructure Navigator
  • App Volumes
  • Horizon Air Hybrid-Mode
  • Integrated OpenStack
  • vCloud Networking and Security
  • vRealize Business for Cloud
  • vRealize Configuration Manager
  • vRealize Hyperic
  • vRealize Networking Insight

These components are not yet compatible with vSphere 6.5. But as we know VMware, they are already working for updates. Please check the VMware Product Interoperability Matrix for further information about updates to the products above.

  • If you have to revert a migration, please check VMware KB2146453 for reverting a vCenter Server to Appliance migration.
  • To roll back a vCenter Server Instance on Windows, please check the vSphere Upgrade Guide.

Upgrade Considerations

Before upgrading your environment, review these critical KB articles to make sure the upgrade will be successful.

vCenter Server

vCenter Server to vCenter Server Appliance

PSC High Availability

ESXi

NSX

vRealize Operations

vSphere Web Client

Known Issues

vCenter Server

vRealize Operations Manager

Security Considerations

TLS protocols

Encryption considerations

  • Running a encrypted KMS virtual machine can cause a loss of data in the event of a host failure.

More details in the VMware Knowledgebase (KB2147548):

https://kb.vmware.com/kb/2147548

*** Update ***

Backup Considerations

There is one thing i missed to mention. If you are using Veeam Availability Suite v9.5 then you can’t do backups with vSphere 6.5 at the moment, because Veeam does not support this vSphere version yet. But also the guys at Veeam are working on an update, which will be (historically) release about two months after general availability of the new vSphere version.

So stay tuned!