In Files

  • rdoc/ruby_lex.rb

Parent

Namespace

Methods

Class/Module Index [+]

Quicksearch

RDoc::RubyLex

Ruby lexer adapted from irb.

The internals are not documented because they are scary.

Public Class Methods

new(content, options) click to toggle source

Creates a new lexer for content. options is an RDoc::Options, only +tab_width is used.

 
               # File rdoc/ruby_lex.rb, line 81
def initialize(content, options)
  lex_init

  if /\t/ =~ content then
    tab_width = options.tab_width
    content = content.split(/\n/).map do |line|
      1 while line.gsub!(/\t+/) {
        ' ' * (tab_width*$&.length - $`.length % tab_width)
      }  && $~
      line
    end.join("\n")
  end

  content << "\n" unless content[-1, 1] == "\n"

  set_input StringIO.new content

  @base_char_no = 0
  @char_no = 0
  @exp_line_no = @line_no = 1
  @here_readed = []
  @readed = []
  @rests = []
  @seek = 0

  @here_header = false
  @indent = 0
  @indent_stack = []
  @lex_state = EXPR_BEG
  @space_seen = false

  @continue = false
  @line = ""

  @skip_space = false
  @readed_auto_clean_up = false
  @exception_on_syntax_error = true

  @prompt = nil
  @prev_seek = nil
  @ltype = nil
end
            
tokenize(ruby, options) click to toggle source

Returns an Array of ruby tokens. See ::new for a description of options.

 
               # File rdoc/ruby_lex.rb, line 64
def self.tokenize ruby, options
  tokens = []

  scanner = RDoc::RubyLex.new ruby, options
  scanner.exception_on_syntax_error = true

  while token = scanner.token do
    tokens << token
  end

  tokens
end
            

Commenting is here to help enhance the documentation. For example, code samples, or clarification of the documentation.

If you have questions about Ruby or the documentation, please post to one of the Ruby mailing lists. You will get better, faster, help that way.

If you wish to post a correction of the docs, please do so, but also file bug report so that it can be corrected for the next release. Thank you.

If you want to help improve the Ruby documentation, please visit Documenting-ruby.org.

blog comments powered by Disqus