The type of business will determine a portion of the equation that answers your question.
Generally, I strongly suggest that whatever technology gets implemented, don't skimp on it. Don't overspend, but don't shoestring it. Get a computer that is robust enough to last for several years (e.g., fast enough, enough storage, etc.)
Take full advantage of online services. One of the things I wish I had implemented from the get go was having my bookkeeping and payroll handled via QuickBooksOnline. I have a bookkeeper to do the former, which is good because I would make a mess of it. I do the payroll myself, and it takes far less time than doing it "manually."
As a marketing and business coach, I have seen how significant a website is for any business. I have also seen very ineffective sites, many of which cost the owner thousands and thousands of dollars. A company's website is so important in my opinion that I give a free web assessment with clear, actionable recommendations for improving site effectiveness.
Having made that point, I do want to say that I'm not talking about SEO when I talk about websites. I believe that SEO is not something most companies need to focus on initially. First, get the marketing message and the action request right...worry about SEO later. Too much money gets spent for too little productive result if this aspect of a website is focused on too soon.
Telephone is a big consideration. Solopreneurs can use cell phones as their business numbers. Services like RingCentral provide a virtual PBX that lets you combine all phones into one central control. Also, services like Skype are great for communicating via phone, especially with overseas contacts.
These are just a few thoughts to start the ball rolling!
-------------------------
Trish Lambert
Helping solo home business owners
stay solo and get successful!
http://www.successinsweatpants.com
Twitter: http://twitter.com/trishlambert
Facebook: http://www.facebook.com/trishlambert1