Adding nerdctl to an OpenShift 4 Windows Node

# Getting a Windows Node on OpenShift
... Not what this post is about, follow the Red Hat and Microsoft documentation and then consult with your certified Red Hat Solutions Architect or Consultant. 

# Now that I have a windows node
*  Administering a windows node is highly discouraged and will probably result in an unsupported configuration.
In order to install nerdctl, log into the necessary Windows Node and switch to PowerShell.
```
ssh win2022

Microsoft Windows [Version 10.0.20348.1070]
(c) Microsoft Corporation. All rights reserved.

administrator@WIN2022 C:\Users\Administrator> powershell
Windows PowerShell
Copyright (C) Microsoft Corporation. All rights reserved.

Install the latest PowerShell for new features and improvements! https://aka.ms/PSWindows

PS C:\Users\Administrator>
```

# Append to System PATH - Permanently
Borrowing commands from a friendly blogger we need to append a new entry to the system PATH.  
[Permanently Modify System Path](https://codingbee.net/powershell/powershell-make-a-permanent-change-to-the-path-environment-variable)
Run these:
```
rem Show Current Path
PS C:\Users\Administrator> $ENV:PATH
rem Save Current Path to a variable
PS C:\Users\Administrator> $oldpath = (Get-ItemProperty -Path 'Registry::HKEY_LOCAL_MACHINE\System\CurrentControlSet\Control\Session Manager\Environment' -Name PATH).path

rem Add a new folder for our binary
PS C:\Users\Administrator> mkdir C:\bin
rem Change directory to the new folder
PS C:\Users\Administrator> cd C:\bin
PS C:\bin> $newpath = "$oldpath;C:\bin"

rem Update System Path
PS C:\bin> Set-ItemProperty -Path 'Registry::HKEY_LOCAL_MACHINE\System\CurrentControlSet\Control\Session Manager\Environment' -Name PATH -Value $newPath

rem Verify the Path
PS C:\bin> (Get-ItemProperty -Path 'Registry::HKEY_LOCAL_MACHINE\System\CurrentControlSet\Control\Session Manager\Environment' -Name PATH).Path

rem Reboot node - or close out of your shells and relogin
Restart-Computer -Force
```

# Download and install nerdctl.exe
With all that yak shaving out of the way we can get to it.
Find the latest windows release (nerdctl-x.y.z-windows-amd64.tar.gz)
[nerdctl-1.0.0-windows-amd64.tar.gz](https://github.com/containerd/nerdctl/releases/download/v1.0.0/nerdctl-1.0.0-windows-amd64.tar.gz)
[nerdctl releases](https://github.com/containerd/nerdctl/releases)
```
rem start Powershell and change to the bin folder
rem Download the release file.
PS C:\bin> curl https://github.com/containerd/nerdctl/releases/download/v1.0.0/nerdctl-1.0.0-windows-amd64.tar.gz -o nerdctl-1.0.0-windows-amd64.tar.gz

rem Extract nerdctl.exe from the compressed file
PS C:\bin> tar -xzvf nerdctl-1.0.0-windows-amd64.tar.gz
x nerdctl.exe

rem Delete the compressed file
PS C:\bin> rm nerdctl-1.0.0-windows-amd64.tar.gz

rem Test a nerdctl.exe command
PS C:\bin> nerdctl.exe ps
CONTAINER ID    IMAGE    COMMAND    CREATED    STATUS    PORTS    NAMES

```


# Celebrate
You can now interact with your containers on the windows node.

Clearing Windows proxy from PowerShell

# The Problem
The Windows Server 2022 Core VM we were working with had a http proxy set but the OpenShift cluster was in an environment that did not need a proxy.  Using the top google hits we were unable to clear the "advproxy" settings.

```
C:\> netsh winhttp show proxy
Current WinHTTP proxy settings:

    Direct access (no proxy server).

C:\> netsh winhttp show advproxy
{
        "Proxy":        "localhost:8080",
        "ProxyBypass":  "*"
}
```

Using `netsh winhttp reset proxy` only cleared the proxy settings.  It did NOT clear the advproxy settings.


# The solution
Create a JSON file with the settings required for `netsh winhttp set advproxy` command.  
```
PS C:\Users\Administrator> New-Item proxy.json
    Directory: C:\Users\Administrator

Mode                 LastWriteTime         Length Name
----                 -------------         ------ ----
-a----        11/14/2022   4:05 PM              0 proxy.json

PS C:\Users\Administrator> Set-Content .\proxy.json '{"Proxy": "", "ProxyBypass": "", "AutoconfigUrl": "", "AutoDetect": false}'
PS C:\Users\Administrator> Get-Content proxy.json
{"Proxy": ""}

```

Then execute the netsh command below:
```
netsh winhttp set advproxy setting-scope=machine settings-file=proxy.json

Current WinHTTP advanced proxy settings:

{
}

```

# Thank You
I do not know the difference between the two settings, just know they both influence the system.

OpenShift 4.11 - Nutanix and Windows Nodes Temporary Workaround

# OpenShift 4.11 and Windows Containers
This feels like a match made in heaven for organizations that are slowly consuming their monoliths or who have decided on Microsoft technologies for their datacenter.  Either way, Windows Node support has come a long way in the last 2 years.

I must preface this discussion with the assumption that "everyone knows that Linux and WIndows are different".  Not in the way "milk is different from water" but more along the lines of steak is different from broccoli.  Sure, there are many ways the interaction feels similar but would probably wouldn't steam a nice steak or smother broccoli in steak sauce(not unreasonable).  Unfortunately, Kubernetes exposes all the ways the two are different.  Some of this is probably to the history of Kubernetes being developed on Linux, using Linux Containers, with the strong drive of kernel developers to maintain a stable ABI.  With WIndows 11 and Windows Server 2022 Microsoft has finally been able to address the ABI/Kernel version issues it suffered from and make the experience closer to Linux.  But there are so many other differences which I won't address.

Next, many organizations are adopting Nutanix in their datacenters and wish to run WIndows Containers on OpenShift on Nutanix.  OpenShift 4.11+ on Nutanx is a wonderful combination and is fully supported.  Windows Container support in Kubernetes is coming along nicely.  Windows Server 2022 makes windows containers usable outside of AKS or Docker Desktop.  And the Red Hat Windows Machine Config Operator(WMCO) automates the process of adding a Windows VM to a Kubernetes cluster with the OpenShift Machine api.  

The WMCO is written to deal with differences between Linux hosts and Windows hosts and is written and maintained by Red Hat and the community.  OpenSSH needs to be installed and enabled on the Windows host as this is the medium which files are transferred, services are created and the host gets configured.  As an operator, WMCO has its own lifecycle and support policy.  As of this writing it only supports platform types of (none, aws and vsphere).

At this point in time the only way to get a Windows Node in a cluster installed on Nutanix is with BYOH and some unsupported steps. 
 Again THIS METHOD IS UNSUPPORTED and support is on its way.  Here is a rough outline too add a host assuming the SDN is OVNKubernetes and Hybrid Networking was configured at cluster install.

# Steps
1. Install cluster with [networking.type: OVNKuberenetes](https://docs.openshift.com/container-platform/4.11/installing/installing_nutanix/installing-nutanix-installer-provisioned.html#installation-configuration-parameters-network_installing-nutanix-installer-provisioned) and [HybridNetworking](https://docs.openshift.com/container-platform/4.11/networking/ovn_kubernetes_network_provider/configuring-hybrid-networking.html)
2. Run [prerequisites](https://docs.openshift.com/container-platform/4.11/windows_containers/enabling-windows-container-workloads.html)
   * install wmco namespace
   * install wmco operator
   * create cloud-private-key secret
      * https://docs.openshift.com/container-platform/4.11/windows_containers/enabling-windows-container-workloads.html
3. Create a VM and install Windows 2022 (core or ui)
4. Make modifications to match the environment.  
5. [Install OpenSSH](https://learn.microsoft.com/en-us/windows-server/administration/openssh/openssh_install_firstuse?tabs=powershell)
6. [Create SSH Key for the Administrator user and add to administrator_authorized_user](https://learn.microsoft.com/en-us/windows-server/administration/openssh/openssh_keymanagement)
   * `$powershell.exe get-content $env:ProgramData\ssh\administrators_authorized_keys`
7. Verify you can log in from your bastion host
8. OpenPort 10250
   * `powershell.exe New-NetFirewallRule -DisplayName "ContainerLogsPort" -LocalPort 10250 -Enabled True -Direction Inbound -Protocol TCP -Action Allow -EdgeTraversalPolicy Allow`
9. Run Windows SysPrep similar to the [vsphere steps 8-9](https://docs.openshift.com/container-platform/4.11/windows_containers/creating_windows_machinesets/creating-windows-machineset-vsphere.html#creating-the-vsphere-windows-vm-golden-image_creating-windows-machineset-vsphere)
10. Shutdown VM and Create a Template or Disk Image
11. Clone VM
12. Start Cloned VM
13. Verify IP is unique and verify reverse DNS (PTR) is correct
14. Change hostname of Cloned VM
   * `powershell.exe Rename-Computer -NewName  -Restart`
   * reboot
15. As documented in the BYOH instructions [Create windows-instances ConfigMap](https://docs.openshift.com/container-platform/4.11/windows_containers/byoh-windows-instance.html)
16. Initially you may see the node come into the cluster correctly.  However, once the kubelet starts it selects the "hybrid interface" IP and tries to join the cluster with this incorrect IP.  
17. Once you see the node added, Ready and Unscheduleable
   * ssh to windows host
   * Display the binpath of the kubelet service
      * `sc.exe qc kubelet`
   * Copy binpath from above command and add  --node-ip= to the end
      * `sc.exe config binPath= "c:\k\kubelet.exe --config=c:\k\kubelet.conf --bootstrap-kubeconfig=c:\k\bootstrap-kubeconfig --kubeconfig=c:\k\kubeconfig --cert-dir=c:\var\lib\kubelet\pki\ --windows-service --logtostderr=false --log-file=C:\var\log\kubelet\kubelet.log --register-with-taints=os=Windows:NoSchedule --node-labels=node.openshift.io/os_id=Windows --container-runtime=remote --container-runtime-endpoint=npipe://./pipe/containerd-containerd --resolv-conf= --cloud-provider= --v=3 --node-ip=192.168.20.104"`
   * Restart kubelet
      * `powershell restart-service kubelet`
18. Now you should see 3+ Approved CSR's and your node should become Ready and Scheduleable

# Troubleshooting:
## Pending CSR's?  
Indicates the reverse DNS record is missing or incorrect
## Node stuck UnScheduleable?  
Node IP isn't in the same subnet as the linux nodes.  Or Firewall is blocking something

# Appendix - Known issues with Windows Containers on Nutanix
## Kubelet chooses the wrong IP to add to the cluster
The kubelet code is failing to lookup the correct IP from the nutanix machine api implementation and falls into the "grab a random IP", which obviously always grabs the wrong one.
This is what step #16 works around.  Providing an argument to the kubelet for the node-ip, bypassing auto-descovery.
## Nutanix Machine Provider API hardcodes "Legacy BIOS" as the boot mode
Customers wishing to boot their kubernetes VM's with UEFI will need to wait for the resolution of  [Issue 29](https://github.com/openshift/machine-api-provider-nutanix/issues/29)
## Nutanix Machine Provider API only accepts "Image"
Do not confuse this with a "Template".  An Image in Nutanix defines just the disk volume (think VMDK or qcow2) and does not contain any VM definitions.
There is not an issue written to address this.
## But I want to try a MachineSet
You can try.  Clone your existing machine set and make the correct modification.  However, the kubelet still chooses the incorrect IP and changes to the service get overwritten by WMCO and the machine-api
## Where can I follow the work
[RFE-3354](https://issues.redhat.com/browse/RFE-3354)
[machine-api-provider-nutanix issue 29](https://github.com/openshift/machine-api-provider-nutanix/issues/29)

Vim and Windows? YES!Vim and Windows? YES!

# Installing vim-console on Windows 2022 from PowerShell
Working with Windows Containers has provided me tremendous education.  Learning PowerShell has shown me how far the windows realm has come in the form of CLI management.  It still seems to be lacking in a few areas, and I really don't like the extremely verbose syntax nor the single dash arguments.  But I digress.  This post is about solving a single problem.  The one problem with endless obtuse workarounds.

## Problem
Editing text files...from a windows CLI (cmd.exe, PowerShell, etc).   

## Solution
Find the Microsoft equivalent of Vim.   Is that Notepad.exe?  Wordpad.exe?  Get-Contents?   
Fail, Fail, Mostly-Fail

## My Solution
Install vim-console; an extremely powerful text editor that has proven to be the staple of all *nix users around the world for decades.

Note: There are many articles similar to[How to Edit Files with a Real PowerShell Text Editor](https://adamtheautomator.com/powershell-text-editor/) that show how to use Chocolaty to install Vim, Emacs or Nano.  However, most of my use cases don't allow Chocolaty in their datacenters.   So, we have to install vim the hard way.

### Steps.
1) SSH to your server.   
Don't know how to setup ssh on Windows?  I will point you at their documentation 
    - [Getting Started OpenSSH for Windows](https://learn.microsoft.com/en-us/windows-server/administration/openssh/openssh_install_firstuse?tabs=powershell)
2) Verify you are in PowerShell 
    Type the command powershell or pwsh and hit enter
        
    Look for the PS in front of your command prompt.
        
3) Download vim-console artifacts [NLUUG - dunno where I found this](https://ftp.nluug.nl/pub/vim/pc/)
    - wget https://ftp.nluug.nl/pub/vim/pc/vim90rt.zip -o vim90rt.zip
    - wget https://ftp.nluug.nl/pub/vim/pc/vim90w32.zip -o vim90w32.zip
4) Extract archives
    - expand-archive .\vim90w32.zip -DestinationPath .
    - expand-archive  -Force .\vim90rt.zip -DestinationPath .
5) Run installer
    - cd vim\vim90
    - run .\install.exe
    - Change settings as needed and say `d` for do it
        

6) Run and Profit
    - Simply type `vim` and hit enter
    - To exit the key sequence `Esc-:-q-` will exit.   

## Conclusion
I didn't go into a lot of detail but hopefully this helps with the steps for using VIM.   

-Nate

Wrangling bash_history across multiple sesions

## BASH_HISTORY_FILE or HISTFILE
I find myself constantly using multiple terminals on the same system.  The default bash history is a good start, but not enough for my needs.  After endless searching, testing, ignoring, deleting, and procrastinating I finally found a solution that works for me.   

1. Session Isolation 
    - I don't need new history items available across sessions
    - Typically the work I am preforming is isolated but similar enough that having commands updated between sessions is both obnoxious and unhelpful.
2. Custom Timestamp 
    - I like seeing a particular timestamp format in my history output (helps me remember what I was doing at that time)
3. Dedupe
    - Like I mentioned above, I don't need commands shared across terminal windows
    - Also, I don't like duplicate uses of command in my history
    - Sure I may lose the first, or prior, occurrence(s) of a command.  But I only require the last use.
4. Write on session exit
    - I don't want to lose my history to a `kill -i `
    - I also don't want to lose my history if I close the wrong session tab
    - dedupe only needs to happen at this point
5. Use only linux bash commands
    - I don't want to download some fancy utility.
    - use the fewest number of commands
6. Unlimited history
    - computers are big enough and fast enough to handle this
    - new versions of bash allow an unlimited history


## `~/.bashrc`

```
HISTCONTROL=ignorespace:ignoredups:erasedups
shopt -s histappend
HISTSIZE=-1
HISTFILESIZE=-1
HISTTIMEFORMAT="[%b %d,%Y %T] "
PROMPT_COMMAND="history -a;${PROMPT_COMMAND}"
```


## `/etc/profile.d/00-dedup-bash-history-on-exit.sh`

```
function deduphistory {
  echo "deduphistory"
  history -w ~/.bash_history.dedup_backup
  history -n; history -a
  tac ~/.bash_history | awk '!x[$0]++' | tac > ~/.bash_history_new_dedup
  history -cr ~/.bash_history_new_dedup
  history -w
}
# cleanup bash history on exit
trap deduphistory EXIT
```


## Profit
Watch your history get appended and deduped on exit.  
No more lost history (mostly)
And be happy


### Files available as gists
https://gist.github.com/therevoman/a7532af575479f0a2b5edfbb0bef8212

Originally posted on github.
https://github.com/therevoman/nate-revo-blog/issues/2 

Welcome to Revo Place

I'm starting a new blog focusing on the technology I love and use everyday.  Come here to join the adventure.

My computer adventure started in the early (19)80's with the TI-99 and various Tandy Computers.  I was the family "Technical Advisor" even though I really only knew just a little more than they did.  Fascinated by what they could offer I was exposed to the Nintendo, ColecoVision, Adam, Amiga 500, Packard Bell, and a Cray Supercomputer.  Being just a youngling I was not allowed to play with the beta edition of Windows NT running on it, but it inspired me to keep my mind open and always be learning.

In the early '90s I entered the BBS scene and opened a BBS on my parents home computer.  Being the encouraging parents they are, and after seeing the first phone bill, they purchased a 2nd line on which I could run so their "ears would be preserved".  

In the mid 90's I was introduced to this novel Operating System named Linux and downloaded the first release on 14 1.44mb floppy diskettes.  Having only 10 disks available I downloaded the first 10 disks during a University class.  That night I began the installation, leaving the computer on and praying there would be no power outage I overwrote the first few diskettes to get the remaining installation images.  Success!  I was presented with a shell prompt of some sort and my lack of knowledge kept me from doing anything more than `startx` and `xeyes`.   

At University I was exposed to Caldera Linux, which I will still hail as the "easiest and most user-friendly" installation ever available.  Too bad their business ethics did what they did (enter groklaw).  

With only one "Java" programming class under my belt I began my career as a web-developer with the goal of using Java on the AS/400 (iseries (is it now series-i?!)) and learning how to install WebSphere v1 and v3 on the platform's IFS.  

The next 18 years I moved around, working in various government and financial institutions.  This gave me exposure to many different organizational structures and to this day helps me relate to the struggles many IT staff experience.  And yes, I believe I can understand the frustration of infrastructure administrators AND developers.  

In 2017 I discovered containers through docker and OpenShift v3.  I had no idea how big of an impact this would have on my career.  In fact, I was so impressed by the technology I jokingly sent in a resume in the hopes I could catch the train.  3 weeks later I arrived at Red Hat Tower in Raleigh, NC for my new hire orientation.  A few weeks after that I was sent to 2 weeks of OpenShift Bootcamp and was released into the wild as a Kubernetes (aka OpenShift) admin.  The journey from Java Developer to OpenShift infrastructure Administrator was enormous, one I am glad I made, and almost overbearing at times.  Thanks to the many great people at Red Hat and my great family I was able to made the transition.  

Now, I spent my free time playing with my son, building many things and dabbling in all sorts of technology.  This blog is dedicated to my Technology adventures and I hope it helps someone else.
  
  -Nate