Spork doesn't seem to run SimpleCov correctly with RSpec and Rails

[fj]

tl;dr:

Summary of results:

summary of issues		Spork
		not running	running
SimpleCov	top of Spork.prefork	works as expected	SimpleCov doesn't run
	top of Spork.each_run	always 0 LOC, 0% coverage	SimpleCov doesn't run

---

I'm on an existing Rails project using Mongoid, Devise, RSpec, and Spork (0.9.0.rc5). Our spec/spec_helper.rb looks like this:

require 'spork'

ENV["RAILS_ENV"] ||= 'test'

Spork.prefork do
  require 'simplecov'
  SimpleCov.start 'rails'

  # trap mongoid
  require 'rails/mongoid'
  Spork.trap_class_method(Rails::Mongoid, :load_models)

  # trap devise
  require 'rails/application'
  Spork.trap_method(Rails::Application, :reload_routes!)

  require File.expand_path("../../config/environment", __FILE__)
  require 'rspec/rails'
  Dir[Rails.root.join("spec/support/**/*.rb")].each {|f| require f}

  DatabaseCleaner.strategy = :truncation

  RSpec.configure do |config|
    config.mock_with :rspec

    config.before(:each) do
      DatabaseCleaner.clean
    end
  end
end

Spork.each_run do
  # reload factories
  Factory.definition_file_paths = [
    File.join(Rails.root, 'spec', 'factories')
  ]
  Factory.find_definitions
end

This configuration will run SimpleCov when Spork is not running, and it produces the expected result. When Spork is running, no coverage message is shown.

If I move the require 'simplecov'; SimpleCov.start 'rails' lines to the beginning of the Spork.each_run block, as SimpleCov's readme suggests, then when Spork is not running, I get this:

Coverage report generated for RSpec to /home/johnf/projects/<elided>/coverage. 0 / 0 LOC (0.0%) covered.

If Spork is running, then instead I get no coverage message, as before.

[fj]

FWIW, I'm running spork with bundle exec spork rspec, rake with bundle exec rake, and rspec with bundle exec rspec --drb spec.

[colszowka]

I'm not using Spork myself, but from what I understand is that it always spins up a new process when the test suite should be run.

The Ruby Coverage library SimpleCov uses works by tracking every required file AFTER it has been loaded. Therefore, if SimpleCov is loaded AFTER the Rails app and all of it's source files, it won't be able to see them (I think this is what you're getting with the each_run solution).

So I imagine the solution would be to make Spork require 'simplecov'; SimpleCov.start before loading the app's environment every time, so the actual code files become visible to SimpleCov. But as I said, I'm not using Spork and I'm not sure how exactly it does it's magic. Maybe the authors of Spork could shed some light onto this?

[colszowka]

Quick update: I pinged @timcharper about this, hopefully he can enlighten us a bit :)

[JeanMertz]

I too am having this issue. Seems not many people run into this problem, as I can't find anything anywhere else.

[turadg]

@fj, I bet your problem is your test environment caches classes. See @estheth's comment in #30.

fwiw, I had been using the hack below in each_run and setting cache_classes to true but that doesn't work:

  # HACK so we can cache-classes in test.rb
  Dir["#{Rails.root}/app/**/*.rb"].each { |f| load f }

  require 'simplecov'
  SimpleCov.start 'rails'

[Godisemo]

I have this issue, i put the two lines for simplecov at the top of the Spork.prefork block. When I use guard no coverage report is generated but when I run rake specit works. Would be nice if it worked with guard though.

[denmarkin]

+1. Same applies to configuration with cucumber. I bet it's spork related

[aantix]

+1. Same config as @Godisemo along with the same issue.

[sevenseacat]

+1 Same here. I swear, it used to work....

[christophermanning]

+1

simplecov (0.5.4)
guard-rspec (0.5.10)
guard-spork (0.4.1)
spork (0.9.0)

[colszowka]

Guys, this obviously is related to #60 - the offender being Spork's requirement to set config.cache_classes = false in config/environments/test.rb. Please try to set it to config.cache_classes = true. As I do not use Spork I don't know what the implications are for spork when you do this, but without it SimpleCov will fail to operate correctly :(

[christophermanning]

@colszowka I had config.cache_classes set to true and false and it still wouldn't generate the coverage directory/report after guard finishes running tests. Additionally setting config.cache_classes = false in config/environments/test.rb causes ActiveRecord::AssociationTypeMismatch exceptions when using FactoryGirl: http://stackoverflow.com/questions/5954690/strange-activerecordassociationtypemismatch

[sunaku]

Something must be wrong in your setups. I successfully use both config.cache_classes=false and FactoryGirl to get code coverage results under Tork (not Spork).

[sunaku]

You guys should try using the Coverage library directly in your setups. If it works, then SimpleCov needs fixing. Else, you know what to do. :)

[timcharper]

I'm afraid I don't have any experience with SimpleCov to be able to shed much light on this. But, I can say this:

You shouldn't need config.cache_classes = false, although that's the easiest way to get your models and such to reload automatically. Read these wiki pages for more info:

https://github.com/sporkrb/spork/wiki/Troubleshooting (specifically: Some changes to files don’t take effect until I restart Spork.)

https://github.com/sporkrb/spork/wiki/Spork.trap_method-Jujutsu

All you should need to do is delay the initialization of SimpleCov until after the process forks, and SimpleCov should be none the wiser.

If a SimpleCov devel wants to do some remove pair programming with me to crack this together, I'd be happy to schedule some time.