VMware

How to reset the ESXi root password?

by

When I check my blog, I can see the last post from February 2022. That’s a long time ago already! Time to write something, isn’t it?

Back in the days when I was working as a Systems Engineer for an IT service provider, it was much easier to write blog posts. Now as a “customer” I don’t find the time or the ideas, or maybe I’m forgetting blog post ideas, not sure why. At least that’s my thought. I’m always struggling if I should blog about this or that, is it worth writing about it, or are there gazillions of blog posts writing about the exact same thing?

Today’s blog post is exactly such a topic, I assume, that has been written about already some times, at least. But it was a problem we had during an ongoing vSphere upgrade project just recently. And I was able to help our operations team to move on with their work. So why not write a blog post about it?

What happened?

As mentioned, we’re currently working on a global vSphere upgrade project. We’ve got many ESXi hosts and clusters all around the world. So far nothing special. And even when there are easy-to-understand guides available internally (I wrote these myself and triple verified), maybe one or the other point on a checklist is forgotten, or you just didn’t think of it in the heat of the moment. One point is “Check the current credentials if they are working”. Thanks to the following troubleshooting guidance, there was no show stopper and only a few minutes of delay for the upgrade of one ESXi host.

The root password for one of the ESXi hosts didn’t work. No chance to log in through the Web UI nor SSH. So what to do then?

There are only two officially supported ways to reset the root password of an ESXi host. You can reinstall the host from scratch or use host profiles. Well, reinstallation would be an option as we’re upgrading vSphere anyways. But this would require some additional time because of the ESXi configuration. Using a host profile can be done, but needs an Enterprise Plus license.

Because we have some spare licenses left for Enterprise Plus (not yet needed for hosts, but already planned to use), we decided to go the way with the host profile. And it wasn’t rocket science!

How can you do it?

The actual troubleshooting chapter is divided into two parts. The first part is changing the current license of an ESXi host, the second part is all about the host profile.

If you don’t have an Enterprise Plus license, then you have to plan reinstalling the ESXi server from scratch.

Change the host license

  1. Log in to the vCenter WebClient (https://yourvcenter.domain.com/ui)
  2. In vCenter, go to Home and then choose Administration and then Licenses
  3. Click the Assets tab and then the HOSTS button
  4. In the Asset column, you can click the filter icon and search for the ESXi host where you want to assign a different license
  5. Select the host, then click Assign License just above the list
  6. Choose the Enterprise Plus license, and click OK
  7. The host will now have an Enterprise Plus license, and you can continue with the steps below.

Remember to switch back the license to the one that was assigned to the ESXi host before.

Extract, change, and apply the host profile

  1. Log in to the vCenter WebClient (https://yourvcenter.domain.com/ui)
  2. In vCenter, go to Home and then choose Policies and Profiles, and click Host Profiles
  3. Click → Extract Host Profile
  4. In the Extract Host Profile menu wizard → Select the host you want to update the password for, then click Next
  5. Name the Host Profile and click Next and then Finish to complete the capture of the host profile template
    • The new host profile should appear on the Host Profile Objects Field
  6. Right Click the new Host Profile and choose → Edit Host Profile
  7. In the Edit Host Profile wizard, uncheck all boxes
  8. Then using the search filter search for → root
  9. Highlight and then select the check box for → User Configuration / root
    • Make sure to only select this item when searching for root
  10. A configurable window will display the root user configuration
  11. At the Password subsection, choose → Fixed password configuration
  12. Here you have to fill in the new password and confirm it before proceeding
  13. Double-check that all other non-applicable boxes have no check marks and proceed to Finish
  14. Once the task completes, right-click the new host profile and choose → Attach/Detach Hosts and Clusters → then select the host in the wizard
  15. Right-click the host profile again, and select Remediate
  16. Remove/detach the host profile from the host
    • At this time the host password should be successfully upgraded

Please be careful. It is recommended that you do this when the host is in maintenance mode. If it is part of a cluster, great. You can move all VMs away from that host with DRS (automatically or manually). If it is a standalone host, make sure to shut down the VMs first, just in case the host reboots. During the writeup, the affected host did not reboot, but there was a checkbox in the remediation settings that could cause the host to reboot.

vExpert Applications are open – Why should you apply?

by

The vExpert applications for 2022 are open now. Nice! But what is this?

Many vendors have special programs for the IT folks around the world. These programs are designed for ambassadors and evangelists. You may get early access to beta versions, marketing information, they help you promote your blog posts and many other great benefits. The VMware vExpert program is VMware’s global evangelism and advocacy program.

There is no certification exam for becoming a vExpert, no course requirement. It is kind of an award or an accreditation. You have to apply for the vExpert program, and a committee will review your application.

Do I have what it takes?

Yes! The vExpert program is about “giving back”. You work in your job, you do great things, and help your users and customers. You’ve got experience in IT technology and working for some years already in that area. You may work at VMware, at a partner, or you may be a VMware customer. Giving back means sharing your knowledge. Did you fix a problem? Blog about it! Did you set up a new solution for a customer? You may speak about it at VMworld! If you’re not a writer, then there are various groups on social media, or you can find forums like the VMTN (and many other non VMware related forums). There are a lot of users, both partners, and customers, asking for help. Help them with your experience and knowledge! You may have written a book about VMware solutions, or you’re focusing on a specific VMware product. Maybe you are a public speaker or a VMUG leader. You have the knowledge, you have experience, share it!

How can I apply?

It is very easy to apply. There are two applications per year. One is starting during the summertime, and one is starting in early December. Both application windows are open for 30 days. After this period, the committee will close the applications and start their voting. This may take up to 45 days. The results for the December applications will be announced in February, the summer application will be announced somewhen in August.

If you’re already a vExpert or even a VCDX, and for sure new applicants have to apply once per year. There are also vExpert sub-programs, like Application Modernization, HCX, Cloud Management, and many others, that you can apply for if you’ve been awarded a vExpert.

The December vExpert applications are open now, and the results will be announced in February 2022. And it might be one of the most eagerly awaited emails, at least if you read about it on some blogs, or see it popping up on Twitter.

Click here to apply for vExpert 2022!

Can I get help?

Yes! VMware announced the vExpert PRO program. There are more than 100 vExpert PROs around the world that are helping you. They can help you with mentoring, tell you what you have to do, and outline the process of the applications. Just reach out to the vExpert PRO close to your area!

What benefits can I get from the vExpert program?

If you check the list on the vExpert website, you can see that there are a lot of benefits. I’m not listing every point here, but I’d like to mention at least some of them.

You will be part of a global network of more than 2000 other vExperts. They are on social media, Twitter, active on blogs and forums. There is also a Slack channel for vExperts. You’ll get a certificate signed by the VMware CEO. One of my personal favorites is the access to 365 day eval licenses for most of the VMware products, like vSphere, vSAN, etc. Throughout the year, you got the opportunity to join private webinars with VMware partners. You will be featured on the vExpert Directory. Also before VMworld, there are pre-launch briefings for bloggers, which means that you may get a sneak peek at some products or announcements. Also, as soon as VMworld will be an in-person event again, there will be vExpert parties both at VMworld US and Europe. And I tell you, they are awesome!

My personal view

I’m working for about 20 years in IT now, including my apprenticeship. I learned many things, and I’m still learning. I had the chance to gain insights into different industries during my work. When I was awarded a vExpert for the first time, I felt honored. It meant a lot to me to be one of only a few hundred to receive such an award. It showed also that my efforts are worth it, and that they have been noticed. I was working on a partner level for many years, helped my customers to build their infrastructure, helped them with troubleshooting. Now, I’m working at a customer, fighting with daily business problems, architecting data centers, migrating workloads, and designing cloud concepts.

Some people kept smiling at my vExpert status because they didn’t believe in certifications, accreditations, etc. But that didn’t bother me. I did my thing. I wrote blog posts, and I still write (even if not as often or regularly as I want). I’m active in forums and help people with troubleshooting. And I’ve always stood up for one thing. I’m grateful for becoming a vExpert, it meant a lot to me, and it still does.

Personally, I have the feeling that, especially in contact with technical support, it enables a different level of communication if the other person recognizes my status as vExpert. A problem may escalate upwards faster, and you may have faster access to 3rd level support. Things like this. But also during projects, when I’m in contact with internal customers and external consultants, they recognize my status and see that I’m not just any IT technician. I know some stuff, I’m experienced, and someone voted for me so I could become a vExpert. As mentioned, it’s not a certification exam you can learn for. It is the personal commitment that shines through here.

No vMotion possible after ESXi host BIOS update

by

I was working on some ESXi upgrades recently. We’re currently preparing everything to make the upgrade to vSphere 7 somewhen smooth as silk. That means that we’re rolling out vSphere 6.7 on all of our systems. Recently, I was tasked to upgrade some hosts in a facility some hundred miles away. The task itself was super easy, managing that with vSphere Update Manager was working like a charm. But before the vSphere upgrade, I had to upgrade the BIOS and server firmware to make sure that we’re fine with the VMware HCL.

The second host was done within one hour and received the complete care package. But the first host took a bit longer due to unforeseen troubleshooting. I’d like to share some helpful tips (hopefully they’re helpful).

What happened?

As mentioned, upgrading the ESXi host through the vSphere Update Manager worked like a charm. But before that, I booted the server remotely with the Service Pack for ProLiant ISO image to upgrade the BIOS and firmware of that server. Also, that went very well and. As there are two ESXi hosts at this location, we had shared storage available and we were able to move the VMs from one host to the other without further issues. One host placed into maintenance mode, upgrade, remove from maintenance mode, and the same for the second server. That was the idea.

Read more

My homelab hardware gets its own rack

by

This project started a long time ago. When I planned the hardware needs for my homelab, I also thought of getting a rack. I had a real IT rack in mind, as you know it from your daily business, maybe back in the days when at least some stuff was on-premises and not everything in the cloud. I wanted to get a small rack with enough space to mount my whole homelab hardware into it, to have a proper cabling solution, and to have flexibility in case my homelab gets an extension.

But that wasn’t easy. There are various flavors of racks. The normal 42 unit IT rack, half-hight racks, and also various wall-mountable racks for patch panels, switches, and smaller devices. I was thinking and tinkering, looking for specs. But in the end, nothing satisfied me. Well, at least not from a price perspective, of being not able to transport it. And then, there was something going on on Twitter:

https://twitter.com/widmerkarl/status/1175392396974145542

Thanks to my colleague Michael Schroeder I’ve found something. He mentioned his IKEA rack, and that made me curious. Earlier in June, my colleague Fred Hofer announced that he moved his hardware into a bigger rack and that it was easier as when he moved from an IKEA Lack rack to the small rack:

https://twitter.com/Fred_vBrain/status/1267580223727550470

And that was the trigger! Why not building my own rack and tailor it to my needs? I don’t have to spend much money on a real IT rack, and I can do something handcrafted. The rack didn’t have to be anything special, there was not much in my personal specification book.

That’s the specifications planned:

  • Small (not full 42 rack units)
  • It should be lightweight
  • Enough space for at least three servers, some switches, and a NAS (or two)
  • Enough space for future homelab upgrades
  • Extensible, if needed
  • Should withstand some weight
  • Wheels!

The idea of building my own IKEA Lack Rack was born.

This whole homelab IKEA Lack Rack story will be covered in a small blog series. This blog post will start the series with some planning stuff, the first pictures, and the BOM, as far as I can provide it already. At least the BOM will be updated if there is a reason for it.

Read more

An easy way to quickly migrate a VMware VM to Synology VMM

by

When it comes to virtualization, I’m working with VMware products in my homelab, alongside (hardware) products from other manufacturers. But some special circumstances made a special solution to a problem necessary. Due to a month of military duty, when I was at home only for the weekend, I shut down my homelab. Not also due to this fact, but also because I’m currently building my own customized rack, where I will install my homelab hardware. Be sure to check my blog frequently to get more information about the rack, as I will blog about it soon!

What’s the reason for this migration?

I’m using Ubiquiti hardware for my networking (lab switches, home networking, including wireless), and also a Pi-Hole as my ad-blocker. These are the only “business-critical” services in my home network. And they were running on my homelab. But what should I do when I shut down everything? Well, VMware Workstation to the rescue! I’m (actually, I was) running an ESXi on VMware Workstation on my gaming computer. This ESXi server was managed with vCenter as a replication target for Veeam Backup and Replication. Quickly migrate the VMs to that virtual ESXi host, and that’s it. But what when I accidentally shut down this PC? Or I want to shut it down? I need another solution which is more like 24/7!

What’s the solution?

That made me think about Synology. I knew that at least some Synology NAS systems can run virtual workloads directly, either as a virtual machine or within Docker. I didn’t want to go with Docker because of the lack of knowledge, and I have only limited system resources on that NAS box. So it will be two VMs running on my Synology box! But how?

You can’t just vmotion your VMware VM to Synology VMM (Virtual Machine Manager). You can export the VMDK files or create an OVF, which you then import into Synology VMM. But that took to long, somehow (in certain circumstances I can be impatient …).

This blog post will show you how you can easily backup your VMware VMs to a Synology box, with their own toolset, and restore it directly into Synology VMM. It might come in handy, in case you’re searching also for a nifty solution to run a Pi-Hole or a Ubiquiti controller. Or some other small VMs.

To be honest, the Synology box isn’t a Ferrari, or a Fright Liner in terms of performance and / or capacity. Such a NAS is always somehow limited in CPU resources and memory. In my case, I was happy that I maxed-out the memory when I initially bought the NAS box. My current NAS looks like this:

You can see, there are not many resources, but it should be fine for some tiny Linux VM. A domain controller can even run on it if the resources are used sparingly. But don’t expect too much… And let’s dive into the topic now.

Read more