It is possible to set up reverse DNS for Azure virtual machines.

Once you obtained a static IP address for your VM and set up forward DNS for it, the steps to be run on a Windows Server VM are as follows.

First, install prerequisites:

Get-Module PowerShellGet -list | Select-Object Name,Version,Path
Install-Module AzureRM
Import-Module AzureRM
Get-Module AzureRM -list | Select-Object Name,Version,Path

The last command verifies the installed Azure module version.

Next, log on to Azure:

Login-AzureRmAccount

This will present a login credentials dialog.

Finally, access the IP Address resource (you can find out its name on the Azure Portal:

$pip = Get-AzureRmPublicIpAddress -Name "<your-ipaddress-resource-name>" `
  -ResourceGroupName "<your-resource-group-name>"
$pip.DnsSettings = New-Object -TypeName `
  "Microsoft.Azure.Commands.Network.Models.PSPublicIpAddressDnsSettings"
$pip.DnsSettings.DomainNameLabel = "<your-domain-name>"
$pip.DnsSettings.ReverseFqdn = "<your-fully-qualified-domain-name>"
Set-AzureRmPublicIpAddress -PublicIpAddress $pip

That should do it. I now have reverse DNS on my Azure VM that I use for test and development.