How to add a new host to a vSAN cluster

This blog post, I call blog posts like these “quick & dirty posts”, will show you today how to add an ESXi host to your vSAN cluster. You may need additional compute power, and/or storage capacity. Or you want to implement another storage policy to leverage storage efficiency and more failures to tolerate (RAID 5 / RAID 6 erasure coding). Maybe you want to create a stretched cluster, which needs an even number of vSAN nodes on each side plus a vSAN witness. So many reasons to add another vSAN node.

But let’s dive into this topic now.

How to add a new host to a vSAN cluster?

In my vSAN cluster, based on the Express Storage Architecture (ESA), I’ve got six hosts currently. I want to add another host to the cluster because of the computing and storage power. How to do this?

Right-click your cluster and choose “Add Hosts…”. So far, nothing special.

Read more

How to shut down a vSAN cluster – and start it again

Just a few weeks ago, to my shame, I stumbled across an interesting feature in VMware vSphere when trying some things with vSAN. To be honest, and to make it clear before we dive into this topic in this blog post: I screwed up when I tested this feature the first time. Because I didn’t know about this feature and because I didn’t proceed as I should have, as per this feature. At the end I had to reinstall all my vSAN nodes and create a new clean environment after that I screwed up. It was somehow needed anyway because of the most recent homelab rebuild. So, somehow a win-win for me and the lab.

So don’t screw up! No, just kidding. you may know the feature better than me. And I can tell you, vSAN is stronger and more resilient than you may think.

In this blog post, I’d like to show you how to shut down a vSAN cluster, and how to start it again. The feature is hidden in plain view, right-click the vSAN cluster and you’re good to go. Or not?

In this blog post, I’m assuming that the vCenter is NOT running on the vSAN cluster. I may update this blog post, or create another one, with vCenter running on the cluster. Without searching the internet and checking the VMware docs, I don’t know by heart if this is even possible. Anyway. So how do you shut down the vSAN cluster?

Read more

New homelab hardware installed – HP Z8 G4 workstation

I may have skipped some homelab generation upgrades in my documentation here. However I have updated the page as far as I could, and I would like to provide you with a brief update on my current homelab setup with this blog post as well.

Last year, my wife and I moved into our own house. Yes, I married my love and we built a house. I didn’t shout about it because it’s something personal and I don’t have to rub it in everyone’s face. But yes, I’m a married house owner now and a loving father. Oh, I forgot to mention that my wife gave birth to a beautiful son this year. So many things happened! But anyway, back to topic.

You may have seen some images I posted on Twitter last year, about the huge IT rack I got my hands on, and the first “production” deployment in my new homelab rack. This “production” deployment was an actual beer fridge that was small enough to fit into that rack. If you don’t believe me, please go ahead and check the pictures here. The beer fridge is still there, but the huge and heavy IT rack has gone. The huge rack has been replaced by a desktop-size rack from StarTech.com. This rack is enough to provide a nice mount for my SuperMicro servers and networking equipment.

But the main topic in this blog post is the recently acquired hardware. I bought a refurbished HP Z8 G4 workstation!

Read more

How to reset the ESXi root password?

When I check my blog, I can see the last post from February 2022. That’s a long time ago already! Time to write something, isn’t it?

Back in the days when I was working as a Systems Engineer for an IT service provider, it was much easier to write blog posts. Now as a “customer” I don’t find the time or the ideas, or maybe I’m forgetting blog post ideas, not sure why. At least that’s my thought. I’m always struggling if I should blog about this or that, is it worth writing about it, or are there gazillions of blog posts writing about the exact same thing?

Today’s blog post is exactly such a topic, I assume, that has been written about already some times, at least. But it was a problem we had during an ongoing vSphere upgrade project just recently. And I was able to help our operations team to move on with their work. So why not write a blog post about it?

What happened?

As mentioned, we’re currently working on a global vSphere upgrade project. We’ve got many ESXi hosts and clusters all around the world. So far nothing special. And even when there are easy-to-understand guides available internally (I wrote these myself and triple verified), maybe one or the other point on a checklist is forgotten, or you just didn’t think of it in the heat of the moment. One point is “Check the current credentials if they are working”. Thanks to the following troubleshooting guidance, there was no show stopper and only a few minutes of delay for the upgrade of one ESXi host.

The root password for one of the ESXi hosts didn’t work. No chance to log in through the Web UI nor SSH. So what to do then?

There are only two officially supported ways to reset the root password of an ESXi host. You can reinstall the host from scratch or use host profiles. Well, reinstallation would be an option as we’re upgrading vSphere anyways. But this would require some additional time because of the ESXi configuration. Using a host profile can be done, but needs an Enterprise Plus license.

Because we have some spare licenses left for Enterprise Plus (not yet needed for hosts, but already planned to use), we decided to go the way with the host profile. And it wasn’t rocket science!

How can you do it?

The actual troubleshooting chapter is divided into two parts. The first part is changing the current license of an ESXi host, the second part is all about the host profile.

If you don’t have an Enterprise Plus license, then you have to plan reinstalling the ESXi server from scratch.

Change the host license

  1. Log in to the vCenter WebClient (https://yourvcenter.domain.com/ui)
  2. In vCenter, go to Home and then choose Administration and then Licenses
  3. Click the Assets tab and then the HOSTS button
  4. In the Asset column, you can click the filter icon and search for the ESXi host where you want to assign a different license
  5. Select the host, then click Assign License just above the list
  6. Choose the Enterprise Plus license, and click OK
  7. The host will now have an Enterprise Plus license, and you can continue with the steps below.

Remember to switch back the license to the one that was assigned to the ESXi host before.

Extract, change, and apply the host profile

  1. Log in to the vCenter WebClient (https://yourvcenter.domain.com/ui)
  2. In vCenter, go to Home and then choose Policies and Profiles, and click Host Profiles
  3. Click → Extract Host Profile
  4. In the Extract Host Profile menu wizard → Select the host you want to update the password for, then click Next
  5. Name the Host Profile and click Next and then Finish to complete the capture of the host profile template
    • The new host profile should appear on the Host Profile Objects Field
  6. Right Click the new Host Profile and choose → Edit Host Profile
  7. In the Edit Host Profile wizard, uncheck all boxes
  8. Then using the search filter search for → root
  9. Highlight and then select the check box for → User Configuration / root
    • Make sure to only select this item when searching for root
  10. A configurable window will display the root user configuration
  11. At the Password subsection, choose → Fixed password configuration
  12. Here you have to fill in the new password and confirm it before proceeding
  13. Double-check that all other non-applicable boxes have no check marks and proceed to Finish
  14. Once the task completes, right-click the new host profile and choose → Attach/Detach Hosts and Clusters → then select the host in the wizard
  15. Right-click the host profile again, and select Remediate
  16. Remove/detach the host profile from the host
    • At this time the host password should be successfully upgraded

Please be careful. It is recommended that you do this when the host is in maintenance mode. If it is part of a cluster, great. You can move all VMs away from that host with DRS (automatically or manually). If it is a standalone host, make sure to shut down the VMs first, just in case the host reboots. During the writeup, the affected host did not reboot, but there was a checkbox in the remediation settings that could cause the host to reboot.

vExpert Applications are open – Why should you apply?

The vExpert applications for 2022 are open now. Nice! But what is this?

Many vendors have special programs for the IT folks around the world. These programs are designed for ambassadors and evangelists. You may get early access to beta versions, marketing information, they help you promote your blog posts and many other great benefits. The VMware vExpert program is VMware’s global evangelism and advocacy program.

There is no certification exam for becoming a vExpert, no course requirement. It is kind of an award or an accreditation. You have to apply for the vExpert program, and a committee will review your application.

Do I have what it takes?

Yes! The vExpert program is about “giving back”. You work in your job, you do great things, and help your users and customers. You’ve got experience in IT technology and working for some years already in that area. You may work at VMware, at a partner, or you may be a VMware customer. Giving back means sharing your knowledge. Did you fix a problem? Blog about it! Did you set up a new solution for a customer? You may speak about it at VMworld! If you’re not a writer, then there are various groups on social media, or you can find forums like the VMTN (and many other non VMware related forums). There are a lot of users, both partners, and customers, asking for help. Help them with your experience and knowledge! You may have written a book about VMware solutions, or you’re focusing on a specific VMware product. Maybe you are a public speaker or a VMUG leader. You have the knowledge, you have experience, share it!

How can I apply?

It is very easy to apply. There are two applications per year. One is starting during the summertime, and one is starting in early December. Both application windows are open for 30 days. After this period, the committee will close the applications and start their voting. This may take up to 45 days. The results for the December applications will be announced in February, the summer application will be announced somewhen in August.

If you’re already a vExpert or even a VCDX, and for sure new applicants have to apply once per year. There are also vExpert sub-programs, like Application Modernization, HCX, Cloud Management, and many others, that you can apply for if you’ve been awarded a vExpert.

The December vExpert applications are open now, and the results will be announced in February 2022. And it might be one of the most eagerly awaited emails, at least if you read about it on some blogs, or see it popping up on Twitter.

Click here to apply for vExpert 2022!

Can I get help?

Yes! VMware announced the vExpert PRO program. There are more than 100 vExpert PROs around the world that are helping you. They can help you with mentoring, tell you what you have to do, and outline the process of the applications. Just reach out to the vExpert PRO close to your area!

What benefits can I get from the vExpert program?

If you check the list on the vExpert website, you can see that there are a lot of benefits. I’m not listing every point here, but I’d like to mention at least some of them.

You will be part of a global network of more than 2000 other vExperts. They are on social media, Twitter, active on blogs and forums. There is also a Slack channel for vExperts. You’ll get a certificate signed by the VMware CEO. One of my personal favorites is the access to 365 day eval licenses for most of the VMware products, like vSphere, vSAN, etc. Throughout the year, you got the opportunity to join private webinars with VMware partners. You will be featured on the vExpert Directory. Also before VMworld, there are pre-launch briefings for bloggers, which means that you may get a sneak peek at some products or announcements. Also, as soon as VMworld will be an in-person event again, there will be vExpert parties both at VMworld US and Europe. And I tell you, they are awesome!

My personal view

I’m working for about 20 years in IT now, including my apprenticeship. I learned many things, and I’m still learning. I had the chance to gain insights into different industries during my work. When I was awarded a vExpert for the first time, I felt honored. It meant a lot to me to be one of only a few hundred to receive such an award. It showed also that my efforts are worth it, and that they have been noticed. I was working on a partner level for many years, helped my customers to build their infrastructure, helped them with troubleshooting. Now, I’m working at a customer, fighting with daily business problems, architecting data centers, migrating workloads, and designing cloud concepts.

Some people kept smiling at my vExpert status because they didn’t believe in certifications, accreditations, etc. But that didn’t bother me. I did my thing. I wrote blog posts, and I still write (even if not as often or regularly as I want). I’m active in forums and help people with troubleshooting. And I’ve always stood up for one thing. I’m grateful for becoming a vExpert, it meant a lot to me, and it still does.

Personally, I have the feeling that, especially in contact with technical support, it enables a different level of communication if the other person recognizes my status as vExpert. A problem may escalate upwards faster, and you may have faster access to 3rd level support. Things like this. But also during projects, when I’m in contact with internal customers and external consultants, they recognize my status and see that I’m not just any IT technician. I know some stuff, I’m experienced, and someone voted for me so I could become a vExpert. As mentioned, it’s not a certification exam you can learn for. It is the personal commitment that shines through here.