Home Classes Methods

Included Modules

Files

README

Class/Module Index

RDoc::Parser::Ruby

Extracts code elements from a source file returning a TopLevel object containing the constituent file elements.

This file is based on rtags

RubyParser understands how to document:

classes
modules
methods
constants
aliases
private, public, protected
private_class_function, public_class_function
module_function
attr, attr_reader, attr_writer, attr_accessor
extra accessors given on the command line
metaprogrammed methods
require
include

Method Arguments

The parser extracts the arguments from the method definition. You can override this with a custom argument definition using the :call-seq: directive:

##
# This method can be called with a range or an offset and length
#
# :call-seq:
#   my_method(Range)
#   my_method(offset, length)

def my_method(*args)
end

The parser extracts yield expressions from method bodies to gather the yielded argument names. If your method manually calls a block instead of yielding or you want to override the discovered argument names use the :yields: directive:

##
# My method is awesome

def my_method(&block) # :yields: happy, times
  block.call 1, 2
end

Metaprogrammed Methods

To pick up a metaprogrammed method, the parser looks for a comment starting with ‘##’ before an identifier:

##
# This is a meta-programmed method!

add_my_method :meta_method, :arg1, :arg2

The parser looks at the token after the identifier to determine the name, in this example, :meta_method. If a name cannot be found, a warning is printed and ‘unknown is used.

You can force the name of a method using the :method: directive:

##
# :method: woo_hoo!

By default, meta-methods are instance methods. To indicate that a method is a singleton method instead use the :singleton-method: directive:

##
# :singleton-method:

You can also use the :singleton-method: directive with a name:

##
# :singleton-method: woo_hoo!

Hidden methods

You can provide documentation for methods that don’t appear using the :method: and :singleton-method: directives:

##
# :method: ghost_method
# There is a method here, but you can't see it!

##
# this is a comment for a regular method

def regular_method() end

Note that by default, the :method: directive will be ignored if there is a standard rdocable item following it.

Public Class Methods

new(top_level, file_name, content, options, stats) click to toggle source

 
               # File rdoc/parser/ruby.rb, line 1469
def initialize(top_level, file_name, content, options, stats)
  super

  @size = 0
  @token_listeners = nil
  @scanner = RDoc::RubyLex.new content, @options
  @scanner.exception_on_syntax_error = false

  reset
end

Public Instance Methods

add_token_listener(obj) click to toggle source

 
               # File rdoc/parser/ruby.rb, line 1480
def add_token_listener(obj)
  @token_listeners ||= []
  @token_listeners << obj
end

collect_first_comment() click to toggle source

Look for the first comment in a file that isn’t a shebang line.

 
               # File rdoc/parser/ruby.rb, line 1488
def collect_first_comment
  skip_tkspace
  res = ''
  first_line = true

  tk = get_tk

  while TkCOMMENT === tk
    if first_line and tk.text =~ %r\A#!/ then
      skip_tkspace
      tk = get_tk
    elsif first_line and tk.text =~ %r\A#\s*-\*-/ then
      first_line = false
      skip_tkspace
      tk = get_tk
    else
      first_line = false
      res << tk.text << "\n"
      tk = get_tk

      if TkNL === tk then
        skip_tkspace false
        tk = get_tk
      end
    end
  end

  unget_tk tk

  res
end

error(msg) click to toggle source

 
               # File rdoc/parser/ruby.rb, line 1520
def error(msg)
  msg = make_message msg
  $stderr.puts msg
  exit(1)
end

extract_call_seq(comment, meth) click to toggle source

Look for a ‘call-seq’ in the comment, and override the normal parameter stuff

 
               # File rdoc/parser/ruby.rb, line 1530
def extract_call_seq(comment, meth)
  if comment.sub!(%r:?call-seq:(.*?)^\s*\#?\s*$/, '') then
    seq = $1
    seq.gsub!(%r^\s*\#\s*/, '')
    meth.call_seq = seq
  end

  meth
end

get_bool() click to toggle source

 
               # File rdoc/parser/ruby.rb, line 1540
def get_bool
  skip_tkspace
  tk = get_tk
  case tk
  when TkTRUE
    true
  when TkFALSE, TkNIL
    false
  else
    unget_tk tk
    true
  end
end

get_class_or_module(container) click to toggle source

Look for the name of a class of module (optionally with a leading: or
with: separated named) and return the ultimate name and container

 
               # File rdoc/parser/ruby.rb, line 1558
  def get_class_or_module(container)
    skip_tkspace
    name_t = get_tk

    # class ::A -> A is in the top level
    if TkCOLON2 === name_t then
      name_t = get_tk
      container = @top_level
    end

    skip_tkspace(false)

    while TkCOLON2 === peek_tk do
      prev_container = container
      container = container.find_module_named(name_t.name)
      if !container
#          warn("Couldn't find module #{name_t.name}")
        container = prev_container.add_module RDoc::NormalModule, name_t.name
      end
      get_tk
      name_t = get_tk
    end
    skip_tkspace(false)
    return [container, name_t]
  end

get_class_specification() click to toggle source

Return a superclass, which can be either a constant of an expression

 
               # File rdoc/parser/ruby.rb, line 1587
def get_class_specification
  tk = get_tk
  return "self" if TkSELF === tk

  res = ""
  while TkCOLON2 === tk or TkCOLON3 === tk or TkCONSTANT === tk do
    res += tk.text
    tk = get_tk
  end

  unget_tk(tk)
  skip_tkspace(false)

  get_tkread # empty out read buffer

  tk = get_tk

  case tk
  when TkNL, TkCOMMENT, TkSEMICOLON then
    unget_tk(tk)
    return res
  end

  res += parse_call_parameters(tk)
  res
end

get_constant() click to toggle source

Parse a constant, which might be qualified by one or more class or module names

 
               # File rdoc/parser/ruby.rb, line 1618
  def get_constant
    res = ""
    skip_tkspace(false)
    tk = get_tk

    while TkCOLON2 === tk or TkCOLON3 === tk or TkCONSTANT === tk do
      res += tk.text
      tk = get_tk
    end

#      if res.empty?
#        warn("Unexpected token #{tk} in constant")
#      end
    unget_tk(tk)
    res
  end

get_constant_with_optional_parens() click to toggle source

Get a constant that may be surrounded by parens

 
               # File rdoc/parser/ruby.rb, line 1638
def get_constant_with_optional_parens
  skip_tkspace(false)
  nest = 0
  while TkLPAREN === (tk = peek_tk) or TkfLPAREN === tk do
    get_tk
    skip_tkspace(true)
    nest += 1
  end

  name = get_constant

  while nest > 0
    skip_tkspace(true)
    tk = get_tk
    nest -= 1 if TkRPAREN === tk
  end
  name
end

get_symbol_or_name() click to toggle source

 
               # File rdoc/parser/ruby.rb, line 1657
def get_symbol_or_name
  tk = get_tk
  case tk
  when  TkSYMBOL
    tk.text.sub(%r^:/, '')
  when TkId, TkOp
    tk.name
  when TkSTRING
    tk.text
  else
    raise "Name or symbol expected (got #{tk})"
  end
end

get_tk() click to toggle source

 
               # File rdoc/parser/ruby.rb, line 1671
def get_tk
  tk = nil
  if @tokens.empty?
    tk = @scanner.token
    @read.push @scanner.get_read
    puts "get_tk1 => #{tk.inspect}" if $TOKEN_DEBUG
  else
    @read.push @unget_read.shift
    tk = @tokens.shift
    puts "get_tk2 => #{tk.inspect}" if $TOKEN_DEBUG
  end

  if TkSYMBEG === tk then
    set_token_position(tk.line_no, tk.char_no)
    tk1 = get_tk
    if TkId === tk1 or TkOp === tk1 or TkSTRING === tk1 then
      if tk1.respond_to?(:name)
        tk = Token(TkSYMBOL).set_text(":" + tk1.name)
      else
        tk = Token(TkSYMBOL).set_text(":" + tk1.text)
      end
      # remove the identifier we just read (we're about to
      # replace it with a symbol)
      @token_listeners.each do |obj|
        obj.pop_token
      end if @token_listeners
    else
      warn("':' not followed by identifier or operator")
      tk = tk1
    end
  end

  # inform any listeners of our shiny new token
  @token_listeners.each do |obj|
    obj.add_token(tk)
  end if @token_listeners

  tk
end

get_tkread() click to toggle source

 
               # File rdoc/parser/ruby.rb, line 1711
def get_tkread
  read = @read.join("")
  @read = []
  read
end

look_for_directives_in(context, comment) click to toggle source

Look for directives in a normal comment block:

#-- - don't display comment from this point forward

This routine modifies it’s parameter

 
               # File rdoc/parser/ruby.rb, line 1724
def look_for_directives_in(context, comment)
  preprocess = RDoc::Markup::PreProcess.new(@file_name,
                                            @options.rdoc_include)

  preprocess.handle(comment) do |directive, param|
    case directive
    when 'enddoc' then
      throw :enddoc
    when 'main' then
      @options.main_page = param
      ''
    when 'method', 'singleton-method' then
      false # ignore
    when 'section' then
      context.set_current_section(param, comment)
      comment.replace ''
      break
    when 'startdoc' then
      context.start_doc
      context.force_documentation = true
      ''
    when 'stopdoc' then
      context.stop_doc
      ''
    when 'title' then
      @options.title = param
      ''
    else
      warn "Unrecognized directive '#{directive}'"
      false
    end
  end

  remove_private_comments(comment)
end

make_message(msg) click to toggle source

 
               # File rdoc/parser/ruby.rb, line 1760
def make_message(msg)
  prefix = "\n" + @file_name + ":"
  if @scanner
    prefix << "#{@scanner.line_no}:#{@scanner.char_no}: "
  end
  return prefix + msg
end

parse_alias(context, single, tk, comment) click to toggle source

 
               # File rdoc/parser/ruby.rb, line 1816
def parse_alias(context, single, tk, comment)
  skip_tkspace
  if TkLPAREN === peek_tk then
    get_tk
    skip_tkspace
  end
  new_name = get_symbol_or_name
  @scanner.instance_eval{@lex_state = EXPR_FNAME}
  skip_tkspace
  if TkCOMMA === peek_tk then
    get_tk
    skip_tkspace
  end
  old_name = get_symbol_or_name

  al = RDoc::Alias.new get_tkread, old_name, new_name, comment
  read_documentation_modifiers al, RDoc::ATTR_MODIFIERS
  if al.document_self
    context.add_alias(al)
  end
end

parse_attr(context, single, tk, comment) click to toggle source

 
               # File rdoc/parser/ruby.rb, line 1768
def parse_attr(context, single, tk, comment)
  args = parse_symbol_arg(1)
  if args.size > 0
    name = args[0]
    rw = "R"
    skip_tkspace(false)
    tk = get_tk
    if TkCOMMA === tk then
      rw = "RW" if get_bool
    else
      unget_tk tk
    end
    att = RDoc::Attr.new get_tkread, name, rw, comment
    read_documentation_modifiers att, RDoc::ATTR_MODIFIERS
    if att.document_self
      context.add_attribute(att)
    end
  else
    warn("'attr' ignored - looks like a variable")
  end
end

parse_attr_accessor(context, single, tk, comment) click to toggle source

 
               # File rdoc/parser/ruby.rb, line 1790
def parse_attr_accessor(context, single, tk, comment)
  args = parse_symbol_arg
  read = get_tkread
  rw = "?"

  # If nodoc is given, don't document any of them

  tmp = RDoc::CodeObject.new
  read_documentation_modifiers tmp, RDoc::ATTR_MODIFIERS
  return unless tmp.document_self

  case tk.name
  when "attr_reader"   then rw = "R"
  when "attr_writer"   then rw = "W"
  when "attr_accessor" then rw = "RW"
  else
    rw = @options.extra_accessor_flags[tk.name]
    rw = '?' if rw.nil?
  end

  for name in args
    att = RDoc::Attr.new get_tkread, name, rw, comment
    context.add_attribute att
  end
end

parse_call_parameters(tk) click to toggle source

 
               # File rdoc/parser/ruby.rb, line 1838
def parse_call_parameters(tk)
  end_token = case tk
              when TkLPAREN, TkfLPAREN
                TkRPAREN
              when TkRPAREN
                return ""
              else
                TkNL
              end
  nest = 0

  loop do
      case tk
      when TkSEMICOLON
        break
      when TkLPAREN, TkfLPAREN
        nest += 1
      when end_token
        if end_token == TkRPAREN
          nest -= 1
          break if @scanner.lex_state == EXPR_END and nest <= 0
        else
          break unless @scanner.continue
        end
      when TkCOMMENT
        unget_tk(tk)
        break
      end
      tk = get_tk
  end
  res = get_tkread.tr("\n", " ").strip
  res = "" if res == ";"
  res
end

parse_class(container, single, tk, comment) click to toggle source

 
               # File rdoc/parser/ruby.rb, line 1873
def parse_class(container, single, tk, comment)
  container, name_t = get_class_or_module(container)

  case name_t
  when TkCONSTANT
    name = name_t.name
    superclass = "Object"

    if TkLT === peek_tk then
      get_tk
      skip_tkspace(true)
      superclass = get_class_specification
      superclass = "<unknown>" if superclass.empty?
    end

    cls_type = single == SINGLE ? RDoc::SingleClass : RDoc::NormalClass
    cls = container.add_class cls_type, name, superclass

    @stats.add_class cls

    read_documentation_modifiers cls, RDoc::CLASS_MODIFIERS
    cls.record_location @top_level

    parse_statements cls
    cls.comment = comment

  when TkLSHFT
    case name = get_class_specification
    when "self", container.name
      parse_statements(container, SINGLE)
    else
      other = RDoc::TopLevel.find_class_named(name)
      unless other
        #            other = @top_level.add_class(NormalClass, name, nil)
        #            other.record_location(@top_level)
        #            other.comment = comment
        other = RDoc::NormalClass.new "Dummy", nil
      end

      @stats.add_class other

      read_documentation_modifiers other, RDoc::CLASS_MODIFIERS
      parse_statements(other, SINGLE)
    end

  else
    warn("Expected class name or '<<'. Got #{name_t.class}: #{name_t.text.inspect}")
  end
end

parse_comment(container, tk, comment) click to toggle source

 
               # File rdoc/parser/ruby.rb, line 1977
def parse_comment(container, tk, comment)
  line_no = tk.line_no
  column  = tk.char_no

  singleton = !!comment.sub!(%r(^# +:?)(singleton-)(method:)/, '\1\3')

  if comment.sub!(%r^# +:?method: *(\S*).*?\n/, '') then
    name = $1 unless $1.empty?
  else
    return nil
  end

  meth = RDoc::GhostMethod.new get_tkread, name
  meth.singleton = singleton

  @stats.add_method meth

  meth.start_collecting_tokens
  indent = TkSPACE.new 1, 1
  indent.set_text " " * column

  position_comment = TkCOMMENT.new(line_no, 1, "# File #{@top_level.file_absolute_name}, line #{line_no}")
  meth.add_tokens [position_comment, NEWLINE_TOKEN, indent]

  meth.params = ''

  extract_call_seq comment, meth

  container.add_method meth if meth.document_self

  meth.comment = comment
end

parse_constant(container, single, tk, comment) click to toggle source

 
               # File rdoc/parser/ruby.rb, line 1923
def parse_constant(container, single, tk, comment)
  name = tk.name
  skip_tkspace(false)
  eq_tk = get_tk

  unless TkASSIGN === eq_tk then
    unget_tk(eq_tk)
    return
  end


  nest = 0
  get_tkread

  tk = get_tk
  if TkGT === tk then
    unget_tk(tk)
    unget_tk(eq_tk)
    return
  end

  loop do
      case tk
      when TkSEMICOLON
        break
      when TkLPAREN, TkfLPAREN, TkLBRACE, TkLBRACK, TkDO
        nest += 1
      when TkRPAREN, TkRBRACE, TkRBRACK, TkEND
        nest -= 1
      when TkCOMMENT
        if nest <= 0 && @scanner.lex_state == EXPR_END
          unget_tk(tk)
          break
        end
      when TkNL
        if (nest <= 0) && ((@scanner.lex_state == EXPR_END) || (!@scanner.continue))
          unget_tk(tk)
          break
        end
      end
      tk = get_tk
  end

  res = get_tkread.tr("\n", " ").strip
  res = "" if res == ";"

  con = RDoc::Constant.new name, res, comment
  read_documentation_modifiers con, RDoc::CONSTANT_MODIFIERS

  if con.document_self
    container.add_constant(con)
  end
end

parse_include(context, comment) click to toggle source

 
               # File rdoc/parser/ruby.rb, line 2010
def parse_include(context, comment)
  loop do
    skip_tkspace_comment

    name = get_constant_with_optional_parens
    context.add_include RDoc::Include.new(name, comment) unless name.empty?

    return unless TkCOMMA === peek_tk
    get_tk
  end
end

parse_meta_method(container, single, tk, comment) click to toggle source

Parses a meta-programmed method

 
               # File rdoc/parser/ruby.rb, line 2025
def parse_meta_method(container, single, tk, comment)
  line_no = tk.line_no
  column  = tk.char_no

  start_collecting_tokens
  add_token tk
  add_token_listener self

  skip_tkspace false

  singleton = !!comment.sub!(%r(^# +:?)(singleton-)(method:)/, '\1\3')

  if comment.sub!(%r^# +:?method: *(\S*).*?\n/, '') then
    name = $1 unless $1.empty?
  end

  if name.nil? then
    name_t = get_tk
    case name_t
    when TkSYMBOL then
      name = name_t.text[1..-1]
    when TkSTRING then
      name = name_t.text[1..-2]
    else
      warn "#{container.top_level.file_relative_name}:#{name_t.line_no} unknown name token #{name_t.inspect} for meta-method"
      name = 'unknown'
    end
  end

  meth = RDoc::MetaMethod.new get_tkread, name
  meth.singleton = singleton

  @stats.add_method meth

  remove_token_listener self

  meth.start_collecting_tokens
  indent = TkSPACE.new 1, 1
  indent.set_text " " * column

  position_comment = TkCOMMENT.new(line_no, 1, "# File #{@top_level.file_absolute_name}, line #{line_no}")
  meth.add_tokens [position_comment, NEWLINE_TOKEN, indent]
  meth.add_tokens @token_stream

  add_token_listener meth

  meth.params = ''

  extract_call_seq comment, meth

  container.add_method meth if meth.document_self

  last_tk = tk

  while tk = get_tk do
    case tk
    when TkSEMICOLON then
      break
    when TkNL then
      break unless last_tk and TkCOMMA === last_tk
    when TkSPACE then
      # expression continues
    else
      last_tk = tk
    end
  end

  remove_token_listener meth

  meth.comment = comment
end

parse_method(container, single, tk, comment) click to toggle source

Parses a method

 
               # File rdoc/parser/ruby.rb, line 2100
def parse_method(container, single, tk, comment)
  line_no = tk.line_no
  column  = tk.char_no

  start_collecting_tokens
  add_token(tk)
  add_token_listener(self)

  @scanner.instance_eval do @lex_state = EXPR_FNAME end

  skip_tkspace(false)
  name_t = get_tk
  back_tk = skip_tkspace
  meth = nil
  added_container = false

  dot = get_tk
  if TkDOT === dot or TkCOLON2 === dot then
    @scanner.instance_eval do @lex_state = EXPR_FNAME end
    skip_tkspace
    name_t2 = get_tk

    case name_t
    when TkSELF then
      name = name_t2.name
    when TkCONSTANT then
      name = name_t2.name
      prev_container = container
      container = container.find_module_named(name_t.name)
      unless container then
        added_container = true
        obj = name_t.name.split("::").inject(Object) do |state, item|
          state.const_get(item)
        end rescue nil

        type = obj.class == Class ? RDoc::NormalClass : RDoc::NormalModule

        unless [Class, Module].include?(obj.class) then
          warn("Couldn't find #{name_t.name}. Assuming it's a module")
        end

        if type == RDoc::NormalClass then
          container = prev_container.add_class(type, name_t.name, obj.superclass.name)
        else
          container = prev_container.add_module(type, name_t.name)
        end

        container.record_location @top_level
      end
    else
      # warn("Unexpected token '#{name_t2.inspect}'")
      # break
      skip_method(container)
      return
    end

    meth = RDoc::AnyMethod.new(get_tkread, name)
    meth.singleton = true
  else
    unget_tk dot
    back_tk.reverse_each do |token|
      unget_tk token
    end
    name = name_t.name

    meth = RDoc::AnyMethod.new get_tkread, name
    meth.singleton = (single == SINGLE)
  end

  @stats.add_method meth

  remove_token_listener self

  meth.start_collecting_tokens
  indent = TkSPACE.new 1, 1
  indent.set_text " " * column

  token = TkCOMMENT.new(line_no, 1, "# File #{@top_level.file_absolute_name}, line #{line_no}")
  meth.add_tokens [token, NEWLINE_TOKEN, indent]
  meth.add_tokens @token_stream

  add_token_listener meth

  @scanner.instance_eval do @continue = false end
  parse_method_parameters meth

  if meth.document_self then
    container.add_method meth
  elsif added_container then
    container.document_self = false
  end

  # Having now read the method parameters and documentation modifiers, we
  # now know whether we have to rename #initialize to ::new

  if name == "initialize" && !meth.singleton then
    if meth.dont_rename_initialize then
      meth.visibility = :protected
    else
      meth.singleton = true
      meth.name = "new"
      meth.visibility = :public
    end
  end

  parse_statements(container, single, meth)

  remove_token_listener(meth)

  extract_call_seq comment, meth

  meth.comment = comment
end

parse_method_or_yield_parameters(method = nil, modifiers = RDoc::METHOD_MODIFIERS) click to toggle source

 
               # File rdoc/parser/ruby.rb, line 2214
def parse_method_or_yield_parameters(method = nil,
                                     modifiers = RDoc::METHOD_MODIFIERS)
  skip_tkspace(false)
  tk = get_tk

  # Little hack going on here. In the statement
  #  f = 2*(1+yield)
  # We see the RPAREN as the next token, so we need
  # to exit early. This still won't catch all cases
  # (such as "a = yield + 1"
  end_token = case tk
              when TkLPAREN, TkfLPAREN
                TkRPAREN
              when TkRPAREN
                return ""
              else
                TkNL
              end
  nest = 0

  loop do
      case tk
      when TkSEMICOLON
        break
      when TkLBRACE
        nest += 1
      when TkRBRACE
        # we might have a.each {|i| yield i }
        unget_tk(tk) if nest.zero?
        nest -= 1
        break if nest <= 0
      when TkLPAREN, TkfLPAREN
        nest += 1
      when end_token
        if end_token == TkRPAREN
          nest -= 1
          break if @scanner.lex_state == EXPR_END and nest <= 0
        else
          break unless @scanner.continue
        end
      when method && method.block_params.nil? && TkCOMMENT
        unget_tk(tk)
        read_documentation_modifiers(method, modifiers)
      end
    tk = get_tk
  end
  res = get_tkread.tr("\n", " ").strip
  res = "" if res == ";"
  res
end

parse_method_parameters(method) click to toggle source

Capture the method’s parameters. Along the way, look for a comment containing:

# yields: ....

and add this as the block_params for the method

 
               # File rdoc/parser/ruby.rb, line 2273
def parse_method_parameters(method)
  res = parse_method_or_yield_parameters(method)
  res = "(" + res + ")" unless res[0] == ((
  method.params = res unless method.params
  if method.block_params.nil?
    skip_tkspace(false)
    read_documentation_modifiers method, RDoc::METHOD_MODIFIERS
  end
end

parse_module(container, single, tk, comment) click to toggle source

 
               # File rdoc/parser/ruby.rb, line 2283
def parse_module(container, single, tk, comment)
  container, name_t = get_class_or_module(container)

  name = name_t.name

  mod = container.add_module RDoc::NormalModule, name
  mod.record_location @top_level

  @stats.add_module mod

  read_documentation_modifiers mod, RDoc::CLASS_MODIFIERS
  parse_statements(mod)
  mod.comment = comment
end

parse_require(context, comment) click to toggle source

 
               # File rdoc/parser/ruby.rb, line 2298
def parse_require(context, comment)
  skip_tkspace_comment
  tk = get_tk
  if TkLPAREN === tk then
    skip_tkspace_comment
    tk = get_tk
  end

  name = nil
  case tk
  when TkSTRING
    name = tk.text
    #    when TkCONSTANT, TkIDENTIFIER, TkIVAR, TkGVAR
    #      name = tk.name
  when TkDSTRING
    warn "Skipping require of dynamic string: #{tk.text}"
    #   else
    #     warn "'require' used as variable"
  end
  if name
    context.add_require RDoc::Require.new(name, comment)
  else
    unget_tk(tk)
  end
end

parse_statements(container, single = NORMAL, current_method = nil, comment = '') click to toggle source

 
               # File rdoc/parser/ruby.rb, line 2324
def parse_statements(container, single = NORMAL, current_method = nil,
                     comment = '')
  nest = 1
  save_visibility = container.visibility

  non_comment_seen = true

  while tk = get_tk do
    keep_comment = false

    non_comment_seen = true unless TkCOMMENT === tk

    case tk
    when TkNL then
      skip_tkspace true # Skip blanks and newlines
      tk = get_tk

      if TkCOMMENT === tk then
        if non_comment_seen then
          # Look for RDoc in a comment about to be thrown away
          parse_comment container, tk, comment unless comment.empty?

          comment = ''
          non_comment_seen = false
        end

        while TkCOMMENT === tk do
          comment << tk.text << "\n"
          tk = get_tk          # this is the newline
          skip_tkspace(false)  # leading spaces
          tk = get_tk
        end

        unless comment.empty? then
          look_for_directives_in container, comment

          if container.done_documenting then
            container.ongoing_visibility = save_visibility
          end
        end

        keep_comment = true
      else
        non_comment_seen = true
      end

      unget_tk tk
      keep_comment = true

    when TkCLASS then
      if container.document_children then
        parse_class container, single, tk, comment
      else
        nest += 1
      end

    when TkMODULE then
      if container.document_children then
        parse_module container, single, tk, comment
      else
        nest += 1
      end

    when TkDEF then
      if container.document_self then
        parse_method container, single, tk, comment
      else
        nest += 1
      end

    when TkCONSTANT then
      if container.document_self then
        parse_constant container, single, tk, comment
      end

    when TkALIAS then
      if container.document_self then
        parse_alias container, single, tk, comment
      end

    when TkYIELD then
      if current_method.nil? then
        warn "Warning: yield outside of method" if container.document_self
      else
        parse_yield container, single, tk, current_method
      end

    # Until and While can have a 'do', which shouldn't increase the nesting.
    # We can't solve the general case, but we can handle most occurrences by
    # ignoring a do at the end of a line.

    when  TkUNTIL, TkWHILE then
      nest += 1
      skip_optional_do_after_expression

    # 'for' is trickier
    when TkFOR then
      nest += 1
      skip_for_variable
      skip_optional_do_after_expression

    when TkCASE, TkDO, TkIF, TkUNLESS, TkBEGIN then
      nest += 1

    when TkIDENTIFIER then
      if nest == 1 and current_method.nil? then
        case tk.name
        when 'private', 'protected', 'public', 'private_class_method',
             'public_class_method', 'module_function' then
          parse_visibility container, single, tk
          keep_comment = true
        when 'attr' then
          parse_attr container, single, tk, comment
        when %r^attr_(reader|writer|accessor)$/, @options.extra_accessors then
          parse_attr_accessor container, single, tk, comment
        when 'alias_method' then
          if container.document_self then
            parse_alias container, single, tk, comment
          end
        else
          if container.document_self and comment =~ %r\A#\#$/ then
            parse_meta_method container, single, tk, comment
          end
        end
      end

      case tk.name
      when "require" then
        parse_require container, comment
      when "include" then
        parse_include container, comment
      end

    when TkEND then
      nest -= 1
      if nest == 0 then
        read_documentation_modifiers container, RDoc::CLASS_MODIFIERS
        container.ongoing_visibility = save_visibility
        return
      end

    end

    comment = '' unless keep_comment

    begin
      get_tkread
      skip_tkspace(false)
    end while peek_tk == TkNL
  end
end

parse_symbol_arg(no = nil) click to toggle source

 
               # File rdoc/parser/ruby.rb, line 2476
def parse_symbol_arg(no = nil)
  args = []
  skip_tkspace_comment
  case tk = get_tk
  when TkLPAREN
    loop do
      skip_tkspace_comment
      if tk1 = parse_symbol_in_arg
        args.push tk1
        break if no and args.size >= no
      end

      skip_tkspace_comment
      case tk2 = get_tk
      when TkRPAREN
        break
      when TkCOMMA
      else
        warn("unexpected token: '#{tk2.inspect}'") if $DEBUG_RDOC
        break
      end
    end
  else
    unget_tk tk
    if tk = parse_symbol_in_arg
      args.push tk
      return args if no and args.size >= no
    end

    loop do
      skip_tkspace(false)

      tk1 = get_tk
      unless TkCOMMA === tk1 then
        unget_tk tk1
        break
      end

      skip_tkspace_comment
      if tk = parse_symbol_in_arg
        args.push tk
        break if no and args.size >= no
      end
    end
  end
  args
end

parse_symbol_in_arg() click to toggle source

 
               # File rdoc/parser/ruby.rb, line 2524
def parse_symbol_in_arg
  case tk = get_tk
  when TkSYMBOL
    tk.text.sub(%r^:/, '')
  when TkSTRING
    eval @read[-1]
  else
    warn("Expected symbol or string, got #{tk.inspect}") if $DEBUG_RDOC
    nil
  end
end

parse_toplevel_statements(container) click to toggle source

 
               # File rdoc/parser/ruby.rb, line 2536
def parse_toplevel_statements(container)
  comment = collect_first_comment
  look_for_directives_in(container, comment)
  container.comment = comment unless comment.empty?
  parse_statements container, NORMAL, nil, comment
end

parse_visibility(container, single, tk) click to toggle source

 
               # File rdoc/parser/ruby.rb, line 2543
def parse_visibility(container, single, tk)
  singleton = (single == SINGLE)

  vis_type = tk.name

  vis = case vis_type
        when 'private'   then :private
        when 'protected' then :protected
        when 'public'    then :public
        when 'private_class_method' then
          singleton = true
          :private
        when 'public_class_method' then
          singleton = true
          :public
        when 'module_function' then
          singleton = true
          :public
        else
          raise "Invalid visibility: #{tk.name}"
        end

  skip_tkspace_comment false

  case peek_tk
    # Ryan Davis suggested the extension to ignore modifiers, because he
    # often writes
    #
    #   protected unless $TESTING
    #
  when TkNL, TkUNLESS_MOD, TkIF_MOD, TkSEMICOLON then
    container.ongoing_visibility = vis
  else
    if vis_type == 'module_function' then
      args = parse_symbol_arg
      container.set_visibility_for args, :private, false

      module_functions = []

      container.methods_matching args do |m|
        s_m = m.dup
        s_m.singleton = true if RDoc::AnyMethod === s_m
        s_m.visibility = :public
        module_functions << s_m
      end

      module_functions.each do |s_m|
        case s_m
        when RDoc::AnyMethod then
          container.add_method s_m
        when RDoc::Attr then
          container.add_attribute s_m
        end
      end
    else
      args = parse_symbol_arg
      container.set_visibility_for args, vis, singleton
    end
  end
end

parse_yield(context, single, tk, method) click to toggle source

 
               # File rdoc/parser/ruby.rb, line 2608
def parse_yield(context, single, tk, method)
  if method.block_params.nil?
    get_tkread
    @scanner.instance_eval{@continue = false}
    method.block_params = parse_yield_parameters
  end
end

parse_yield_parameters() click to toggle source

 
               # File rdoc/parser/ruby.rb, line 2604
def parse_yield_parameters
  parse_method_or_yield_parameters
end

peek_read() click to toggle source

 
               # File rdoc/parser/ruby.rb, line 2616
def peek_read
  @read.join('')
end

peek_tk() click to toggle source

Peek at the next token, but don’t remove it from the stream

 
               # File rdoc/parser/ruby.rb, line 2623
def peek_tk
  unget_tk(tk = get_tk)
  tk
end

read_directive(allowed) click to toggle source

Directives are modifier comments that can appear after class, module, or method names. For example:

def fred # :yields: a, b

or:

class MyClass # :nodoc:

We return the directive name and any parameters as a two element array

 
               # File rdoc/parser/ruby.rb, line 2640
def read_directive(allowed)
  tk = get_tk
  result = nil
  if TkCOMMENT === tk
    if tk.text =~ %r\s*:?(\w+):\s*(.*)/
      directive = $1.downcase
      if allowed.include?(directive)
        result = [directive, $2]
      end
    end
  else
    unget_tk(tk)
  end
  result
end

read_documentation_modifiers(context, allow) click to toggle source

 
               # File rdoc/parser/ruby.rb, line 2656
def read_documentation_modifiers(context, allow)
  dir = read_directive(allow)

  case dir[0]
  when "notnew", "not_new", "not-new" then
    context.dont_rename_initialize = true

  when "nodoc" then
    context.document_self = false
    if dir[1].downcase == "all"
      context.document_children = false
    end

  when "doc" then
    context.document_self = true
    context.force_documentation = true

  when "yield", "yields" then
    unless context.params.nil?
      context.params.sub!(%r(,|)\s*&\w+/,'') # remove parameter &proc
    end

    context.block_params = dir[1]

  when "arg", "args" then
    context.params = dir[1]
  end if dir
end

remove_private_comments(comment) click to toggle source

 
               # File rdoc/parser/ruby.rb, line 2685
def remove_private_comments(comment)
  comment.gsub!(%r^#--\n.*?^#\+\+/, '')
  comment.sub!(%r^#--\n.*/, '')
end

remove_token_listener(obj) click to toggle source

 
               # File rdoc/parser/ruby.rb, line 2690
def remove_token_listener(obj)
  @token_listeners.delete(obj)
end

reset() click to toggle source

 
               # File rdoc/parser/ruby.rb, line 2694
def reset
  @tokens = []
  @unget_read = []
  @read = []
end

scan() click to toggle source

 
               # File rdoc/parser/ruby.rb, line 2700
  def scan
    reset

    catch(:eof) do
      catch(:enddoc) do
        begin
          parse_toplevel_statements(@top_level)
        rescue Exception => e
          $stderr.puts "

RDoc failure in #{@file_name} at or around line #{@scanner.line_no} column
#{@scanner.char_no}

Before reporting this, could you check that the file you're documenting
compiles cleanly--RDoc is not a full Ruby parser, and gets confused easily if
fed invalid programs.

The internal error was:

"

          e.set_backtrace(e.backtrace[0,4])
          raise
        end
      end
    end

    @top_level
  end

skip_for_variable() click to toggle source

skip the var [in] part of a ‘for’ statement

 
               # File rdoc/parser/ruby.rb, line 2773
def skip_for_variable
  skip_tkspace(false)
  tk = get_tk
  skip_tkspace(false)
  tk = get_tk
  unget_tk(tk) unless TkIN === tk
end

skip_method(container) click to toggle source

 
               # File rdoc/parser/ruby.rb, line 2781
def skip_method(container)
  meth = RDoc::AnyMethod.new "", "anon"
  parse_method_parameters(meth)
  parse_statements(container, false, meth)
end

skip_optional_do_after_expression() click to toggle source

while, until, and for have an optional do

 
               # File rdoc/parser/ruby.rb, line 2734
def skip_optional_do_after_expression
  skip_tkspace(false)
  tk = get_tk
  case tk
  when TkLPAREN, TkfLPAREN
    end_token = TkRPAREN
  else
    end_token = TkNL
  end

  nest = 0
  @scanner.instance_eval{@continue = false}

  loop do
    case tk
    when TkSEMICOLON
      break
    when TkLPAREN, TkfLPAREN
      nest += 1
    when TkDO
      break if nest.zero?
    when end_token
      if end_token == TkRPAREN
        nest -= 1
        break if @scanner.lex_state == EXPR_END and nest.zero?
      else
        break unless @scanner.continue
      end
    end
    tk = get_tk
  end
  skip_tkspace(false)

  get_tk if TkDO === peek_tk
end

skip_tkspace(skip_nl = true) click to toggle source

Skip spaces

 
               # File rdoc/parser/ruby.rb, line 2790
def skip_tkspace(skip_nl = true)
  tokens = []

  while TkSPACE === (tk = get_tk) or (skip_nl and TkNL === tk) do
    tokens.push tk
  end

  unget_tk(tk)
  tokens
end

skip_tkspace_comment(skip_nl = true) click to toggle source

Skip spaces until a comment is found

 
               # File rdoc/parser/ruby.rb, line 2804
def skip_tkspace_comment(skip_nl = true)
  loop do
    skip_tkspace(skip_nl)
    return unless TkCOMMENT === peek_tk
    get_tk
  end
end

unget_tk(tk) click to toggle source

 
               # File rdoc/parser/ruby.rb, line 2812
def unget_tk(tk)
  @tokens.unshift tk
  @unget_read.unshift @read.pop

  # Remove this token from any listeners
  @token_listeners.each do |obj|
    obj.pop_token
  end if @token_listeners
end

warn(msg) click to toggle source

 
               # File rdoc/parser/ruby.rb, line 2822
def warn(msg)
  return if @options.quiet
  msg = make_message msg
  $stderr.puts msg
end

Commenting is here to help enhance the documentation. For example, sample code, or clarification of the documentation.

If you are posting code samples in your comments, please wrap them in "<pre><code class="ruby" > ... </code></pre>" markup in order to get syntax highlighting.

If you have questions about Ruby or the documentation, please post to one of the Ruby mailing lists. You will get better, faster, help that way.

If you wish to post a correction of the docs, please do so, but also file a bug report so that it can be corrected for the next release. Thank you.