Defining #method_missing and #respond_to? at the same time

I was reading Eloquent Ruby yesterday morning (buy a copy if you haven’t already), and it got me thinking about one of my “favorite” Ruby gotchas: defining #method_missing without a corresponding #respond_to?. E.g.:

class Liar
  def method_missing(*args)
    "Oops, I lied"
  end
end

l = Liar.new
l.respond_to?(:foo) # => false
l.foo # => "Oops, I lied"

The resulting code violates the Principle of Least Surprise, and often interacts in unexpected ways with other libraries.

I started wondering if it would be possible to define #method_missing and #respond_to? at the same time, at least for common #method_missing idioms. After some fiddling I came up with something that worked pretty well. Here’s how it looks:

class Foo
  extend MatchMethodMacros

  match_method(/Amake_me_a_/) do |name, *args|
    food = /Amake_me_a_(.*)$/.match(name.to_s)[1]
    "Make your own damn #{food}"
  end

  match_method(/Asudo_make_me_a/) do |name, *args, &block|
    food = /Asudo_make_me_a_(.*)$/.match(name.to_s)[1]
    "Coming right up, one #{food}"
  end

  def method_missing(name, *args)
    # match_method uses modules, so we can use super to delegate to
    # the generated #method_missing definitions.
    super
  rescue NoMethodError
    "We don't do that kind of thing here"
  end
end

foo = Foo.new

foo.respond_to?(:fix_me_a_sandwich) # => false
foo.respond_to?(:make_me_a_sandwich) # => true
foo.respond_to?(:sudo_make_me_a_sandwich) # => true

foo.fix_me_a_sandwich # => "We don't do that kind of thing here"
foo.make_me_a_sandwich # => "Make your own damn sandwich"
foo.sudo_make_me_a_sandwich # => "Coming right up, one sandwich"

And here’s the implementation:

module MatchMethodMacros
  def match_method(matcher, &method_body) 
    mod = Module.new do
      define_method(:method_missing) do |method_name, *args|
        if matcher === method_name.to_s
          instance_exec(method_name, *args, &method_body)
        else
          super(method_name, *args)
        end
      end

      define_method(:respond_to_missing?) do |method_name, include_private|
        # Even though this is in the #respond_to_missing? hook we
        # still need to call 'super' in case there are other included
        # modules which also define #respond_to_missing?
        (matcher === method_name) || super(method_name, include_private)
      end
    end
    include mod
  end
end

(Also available as a Gist)

It turned out to be a relatively straightforward bit of metaprogramming. Breaking it down, it works like this:

  1. match_method is a “macro” – a method intended to be used at the class or module level to define other methods.
  2. match_method takes a matcher (anything which responds to ===, such as a Regexp) and a block. The matcher determines if the missing method name has been matched. The block becomes the body of the method.
  3. An anonymous module is created to house the new methods. Putting the methods inside their own module makes it possible to make multiple calls to match_method without each one overwriting the last one’s work, as well as for the client class to also define its own explicit method_missing.
  4. Inside the anonymous module, a new #method_missing is defined. It uses the matcher to determine if the method being called is a match, and if so, it triggers the method_body block to be called in the context of the instance. Otherwise it passes to the next #method_missing (which will be Ruby’s default #method_missing if nothing else).
  5. A method #respond_to_missing? is also defined, which simply checks to see if the matcher matches the given method name in String form. If not it passes to the next #respond_to_missing? using super. Note that Ruby doesn’t allow use of the bare version of super (which passes the original arguments along) inside a method defined with define_method. Instead I have to explicitly passa the arguments along.

    #respond_to_missing? is the Ruby 1.9 way of hooking into respond_to?. Ordinarily it would free us from the need to invoke super at all, because #respond_to? does that before checking #respond_to_missing?. But in this case we may have multiple definitions of #respond_to_missing? defined in different match_method-generated modules all included in the same class, and the super is required to invoke all of them.

    If none of that made sense, I don’t blame you. This stuff sometimes hurts my head.

  6. Finally, the generated module with its #method_missing and #respond_to_missing? methods is included into the invoking class.

The only obvious downside of this approach is that there’s no way I can find to pass a block into the instance_exec, so even though Ruby 1.9 allows passing blocks into blocks, it’s not possible to write #match_method methods which take blocks.

In the words of Joel Hodgson: what do you think, sirs?

This entry was posted in Stupid Ruby Tricks and tagged , , , . Bookmark the permalink.
  • http://twitter.com/saturnflyer Jim Gay

    Very nice. It strikes me that this reveals a failure of the language. When would you intentionally *not* want to define both at the same time? Yet I’d also not want to have to think about intentionally defining both at the same time. It should just work.

    • http://profiles.google.com/davearonson Dave Aronson

      Yes, that did strike me as extra overhead of exactly the kind I’d expect Ruby to save us from.  Unifying the creation of both methods makes sense, but another way sprang to my mind.  That is to have both method_missing and respond_to_missing? do nothing but call a third method, which would take an additional boolean flag to say whether we’re just checking, or we want to *execute* the response code.  Avdi’s way is certainly more useful for making on-the-fly chains of “responsiveness” (for lack of a better term), a la Decorator Pattern, but I’ll still try to code up an example of my idea and see if I can find some way to make use of it.  :-)

      • http://avdi.org Avdi Grimm

        I like where you’re going with that. I had a very similar thought, and I’d like to see how it pans out.

        • http://ngauthier.com Nick Gauthier

          Yeah that is cool. Reminds me of the “public method private method” style from tail recursion solutions.

        • http://profiles.google.com/davearonson Dave Aronson

          Okay, here’s a quick and very dirty example:

          ===8<— cut here—
          #! /Users/dave/.rvm/rubies/ruby-1.9.3-head/bin/ruby
          # adjust the above as needed; on my system
          # /usr/bin/ruby is 1.8.7 so no r_t_m?.  :-(

          class Bones

            def method_missing sym, *args, &block
              check_or_do true, sym, *args, &block
            end

            def respond_to_missing? sym, include_private
              check_or_do false, sym, nil, nil
            end

            private

            def check_or_do doit, sym, *args, &block
              str = sym.to_s
              parts = str.split '_'
              print "#{doit ? 'Attempting' : 'Checking'} "#{parts.join ' '}": "
              new_args = parts.concat args
              if /^heal_.*/.match str
                doit ? heal(new_args) : true
              elsif /^set_.*/.match str
                doit ? set(new_args) : true
              else
                doit ? puts("Dammit, Jim, I'm a doctor, not a whatever!") : false
              end
            end

            def heal disease
              puts "Healing #{disease.drop(1).join ' '}"
            end

            def set bone
              puts "Setting #{bone.drop(1).join ' '}"
            end

          end

          b = Bones.new
          puts b.respond_to? :heal_me
          puts b.respond_to? :set_humerus
          puts b.respond_to? :lay_bricks
          b.heal_Spocks :seven_year_itch
          b.set_my_fractured :radius_and_ulna
          b.break_laws_of_physics reason: "to get more power to the engines"
          ===8<—cut here—

          On my system, running this yields:

          ===8<—cut here—
          Checking "heal me": true
          Checking "set humerus": true
          Checking "lay bricks": false
          Attempting "heal Spocks": Healing Spocks seven_year_itch
          Attempting "set my fractured": Setting my fractured radius_and_ulna
          Attempting "break laws of physics": Dammit, Jim, I'm a doctor, not a whatever!

          ===8<—cut here—

          I don't like the repetition of "if these are the droids we're looking for, doit ? grab them : say yes".  If, as in this example, the arg passing is all done the same way, perhaps it could be boiled down to a hash of matchers and their corresponding methods… but I didn't want to restrict it like that for the general case.  Then again, there's much to be said for keeping method_missing very very simple, and relying on shims to do any arg tweaking needed — as Russ Olsen pointed out at NoVaRUG's Newbie On Rails meeting last night, if you're going to make mistakes, *don't* make them in method_missing!  :-)  (On the other claw, I suppose that applies to anything *called by* m_m as well….)

        • http://profiles.google.com/davearonson Dave Aronson

          And here’s another whack at it, that handles dynamic situations:

          ===8(disease,*args){ puts “Healing #{disease.join ‘ ‘}” }
          b.add_pseudomethod ‘set’, ->(bone,*args){ puts “Setting #{bone.join ‘ ‘}” }

          puts b.respond_to? :heal_me
          puts b.respond_to? :set_humerus
          puts b.respond_to? :lay_bricks
          b.heal_Spocks :seven_year_itch
          b.set_my_fractured :radius_and_ulna
          b.break_laws_of_physics reason: “to get more power to the engines”
          ===8<— cut here—

          Again, running it yields (no block-pun intended):

          ===8<— cut here—
          Checking "heal me": true
          Checking "set humerus": true
          Checking "lay bricks": false
          Attempting "heal Spocks": Healing Spocks seven_year_itch
          Attempting "set my fractured": Setting my fractured radius_and_ulna
          Attempting "break laws of physics": Dammit, Jim, I'm a doctor, not a whatever!
          ===8<— cut here—

          It's very restrictive on the args, but another iteration could come up
          with some way to store and retrieve what the arg list needs to look like
          (including presence/absence of a block), and construct the list
          appropriately.  Later….

  • http://ngauthier.com Nick Gauthier

    When I’m defining method missing, it boils down to two cases:

    1) I have a finite number of known methods that I am using metaprogramming to define quickly
    2) Runtime configurable methods need to be handled intelligently. I don’t know how many or what they look like

    The matcher-style here is similar to #1. When this happens to me I try to avoid using method_missing because it’s cleaner to have respond_to work and have public_methods work as well. In this case, I use metaprogramming to really define the methods, not use method_missing.

    In the case of #2, I’d be more interested in code that dynamically defines and undefines methods when the object data changes. So for a Rails ActiveRecord attributes method, whenever #attributes changes, unload the anonymous Attributes module, then define a new Attributes module and make a getter/setter for each attribute.

    Then respond_to, public_methods, and friends will just work.

    This would change your dsl to something like:

    can_not_make :sandwich
    can_make :sandwich, :with => :sudo

    each of those would define methods on an anonymously included module, as opposed to defining a method_missing on a different module each time.

    Also, in addition to being more rubyish and inspectable, it would be faster because it doesn’t have to scan a bunch of regexes whenever you want to call a dynamic method.

    xoxo ngauthier

    • http://avdi.org Avdi Grimm

      Yeah, see the category this was filed under :-) This is definitely a “just because you can, doesn’t necessarily mean you should” type of hack.

      • http://ngauthier.com Nick Gauthier

        Yeah totally. But I get off on stepping back and attacking problems in different ways, so you’ll have to humor my rantings and ravings :-P

      • http://twitter.com/dennismajor1 Dennis Major

        Even if it is a “Stupid Ruby Trick”, I’m glad you “could and you did” – as someone new to metaprogramming I learned a lot from working my way through  this “hack” – thanks for taking the time.  

        • http://avdi.org Avdi Grimm

          You’re welcome, it was fun :-)

  • http://twitter.com/myronmarston Myron Marston

    There’s one common use of `method_missing` where this isn’t really even an issue: proxy objects.  If you have a proxy object that uses method_missing to proxy to a target object, and the proxy does not have a respond_to? definition (i.e. because you’ve subclassed BasicObject, or whatever), then respond_to? itself is a missing method that will be proxied to the target and behave just fine.

    I was working on a proxy object once and hadn’t yet realized this.  I wrote a test for `respond_to?` and expected it to fail and when it didn’t, I realized I didn’t need to implement it :).

    • http://avdi.org Avdi Grimm

      Good point!

  • Devin Ben-Hur

    Nice bit of meta trickery Avdi. I’m with Jim that this divorce between method missing and a respond_to? always seemed like a language wart.

    Regarding your last point about not being able to pass a block into the instance exec’d block, I found a kludgy work around; add the block from the method_missing args to the instance_exec args, and have the called block inspect its last arg and treat it as the block if it’s a Proc: https://gist.github.com/1445597

  • http://twitter.com/peeja Peter Jaros

    That looks nifty, for the cases it fits (which is a lot of them).

    Here’s a semi-philisophical questions I’ve hit recently: is it okay for an object to conditionally respond to a message?  Is it okay for an object to start or stop responding to a message during its lifetime?

    • http://avdi.org Avdi Grimm

      I’d say yes. Consider the GoF state pattern.

      • Peter Jaros

        Well, the state pattern lets an object change the way it responds to a message over its lifetime, but it doesn’t change what messages it responds to, does it?  It can change the implementation, but not the interface.  That’s always how I’ve understood it; is that a limited view of the pattern?

        • http://avdi.org Avdi Grimm

          It depends on what you mean by “responding to a message”. The State pattern can legitimately switch a method from doing what it says to raising NotImplementedError (whatever the language-local version of that error is), to indicate the caller has no business calling that method in that state. That’s a lot like an object stopping responding to a method, although technically it might still report that it responds to that message. So it kind of depends on your point of view.

        • http://profiles.google.com/davearonson Dave Aronson

          As Avdi wrote, it depends on your point of view.  My usual example of a real-ilfe state machine is a vending machine.  Certainly different states *seem to* respond to different messages.  Suppose you select an item but you haven’t put in enough money.  The input is not valid for that state.  Some will “respond” with an error message to that effect, while others will *seem to* not respond at all.  The former could be seen as analogous to what Ruby does when the object *doesn’t* respond (i.e., raise “no method”), or to a properly caught exception (presenting a reasonable error message and fixing anything that went wrong).  IMHO it leans more towards the latter, as letting the exception propagate all the way down essentially crashes it, and obviously that would be Very Bad Design of a vending machine!  The latter could be seen as analogous to a Ruby object responding with an apparently no-op method, or again a properly caught exception.

    • http://profiles.google.com/davearonson Dave Aronson

      In addition to State Pattern, there’s DCI.  I’m not sure I grok it in fullness, just having attended one talk on it, but a large part of it seems to be about dynamically adding methods to a single object, to enable capabilities that it doesn’t always need.  Suddenly the object responds to things it didn’t before.

      What sorts of conditions did you have in mind?

  • Matthew Margolis

    Very cool.  I have been hacking on a little gem over the last few weeks to explore this same question.  https://github.com/mrmargolis/method_missing_router  I like your module based approach, it’s quite different than what I ended up with.

  • http://twitter.com/mbleigh Michael Bleigh

    Wrote about something similar recently, except I was using `respond_to?` behavior to define whether `method_missing` was a hit or miss: http://intridea.com/2011/11/16/dry-magic-methods

    Basically taking `respond_to?` as a way to make `method_missing` more confident.

  • jrochkind

    Nice… but why would you want to rescue NoMethodError?

    In addition to not defining respond_to? to match, implementations of method_missing that just swallow messages they _don’t_ know how to do anything with are equally peeving. Isn’t raising the right thing to do there, so whether functionality is defined with method_missing or ordinary methods remains an implementation detail invisible to the outside observer, including in cases of unrecognized messages? (Don’t get me started on NameError vs NoMethodError, but that’s ruby’s existing problem).

    • http://avdi.org Avdi Grimm

      I think you may want to read the examples again. The rescue NoMethodError was just to demonstrate that an explicit method_missing is respected. It’s not part of the main code being demonstrated.