That depends on the following:

1. Your CSS (and front-end) skills/knowledge
If you have the capacity to build the front-end yourself, then you might not need Bootstrap.

2. The design work
Do you have to build a site from a given design? Is that design vastly different from what Bootstrap or a Bootstrap theme looks like? If you're building something vastly different from Bootstrap's styles, and you decide to go with Bootstrap, you're going to have to do a lot of customisation to the existing Bootstrap components. If you skip Bootstrap you can build a fit-for-purpose front-end.

3. Your motivation for building the site
If you need to quickly prototype a website, build an alpha or ship something quickly, then Bootstrap is great for that.
If you're building a site for the long haul the main reason you'd want to use Bootstrap is for developer familiarity.

4. The components
Does Bootstrap have the necessary components you need for the site you want to build? If there's not a lot of cross-over between the patterns you need and what Bootstrap provides, it might not be worth the effort.

5. Accessibility and Semantic HTML concerns
Bootstrap has improved a lot in terms of Semantic HTML and accessibility, but it's still got a way to go. If Accessibility and Semantic HTML is important to you, you're going to have to do a lot of customisations.

6. Web Performance
If you go with Bootstrap, you're going to have to make a concerted effort to cut away all the components you don't need. You don't want to incorporate the full libraries if you're not going to use all of it. On top of that, a lot of the Bootstrap scripts require jQuery. If your site is basic enough, you don't need jQuery.

7. Forgoing other front-end frameworks
Bootstrap isn't the only front-end framework. You can choose from:
Foundation, Semantic UI, etc etc. Are you making the right choice?

8. Forgoing the opportunity to learn through doing
Like using a calculator can make you lazy with maths, Bootstrap makes you lazy with CSS and HTML. All the hard problems have been abstracted away, and you'll remain none-the-wiser. If you have the experience, or you don't care, then Bootstrap is a wonderful tool to punch out websites.

Hope that helps!