The machines
Every Halfwild server is the same: eight NVIDIA A100 80GB GPUs joined by NVLink in one data center chassis. One server goes to one customer.
Specification
| GPUs | 8× NVIDIA A100 80GB SXM4 |
|---|---|
| GPU memory | 640 GB in total |
| Interconnect | NVLink and NVSwitch, 600 GB/s per GPU |
| Processors | 2× AMD EPYC, 64 cores or more |
| Memory | 1 TB or more |
| Storage | 15 TB or more of local NVMe |
| Network | Dedicated uplink with public IPv4 |
| Access | Root over SSH and an out-of-band console |
| Software | Ubuntu with NVIDIA drivers, CUDA and Docker, or an operating system of your choice |
| Power | About 6 kW per server |
The exact processors, memory, storage and network of your server are written into your quote before you sign.
Why the A100
With 640 GB of GPU memory on one machine, an A100 server fine-tunes and serves most open models without splitting them across hosts. Its software support is mature, and at about 6 kW it fits a standard data center rack. That keeps the cost of running it, and so your price, lower than newer hardware. If your work needs H100s or more than one server, ask; we will quote it or tell you who can.
Where they run
The servers sit in a staffed colocation data center in the Tampa area, with redundant power and cooling and engineers on site. We'll name the facility in your quote, and you are welcome to visit.
To begin, write to [email protected] with what you are training or serving and when you need it, or use the contact form. You will hear back from me within one business day.