Staples offers the Swingline Standard Stapler Value Pack for $5. Check out via auto-restock to cut it to $4.75 and bag free shipping. That's the lowest price we could find by $4. The pack includes a stapler, stapler remover, and 1,250 staples.
Be sure to cancel your subscription after your order ships to avoid being billed for subsequent shipments.