Project

General

Profile

« Previous | Next » 

Revision 34bef607

Added by Marc Dequènes over 13 years ago

  • ID 34bef607d2fb02b95306234d4e8fd03d7450fa92

[evol] cleanup and preliminary work on the Postman (and Needle-ization)

View differences:

IDEAS
all bots:
- API:
* list manipulators:
+ /? -> search(<criterias>)
+ /+ -> create(<data>)
+ /- -> delete(<name>)
* node def:
+ name
+ array of names
+ regex: matching names
+ nil: method_missing-like catchall
* client API:
+ exists?
+ call
+ [<name>]: sub-node
* define_interface:
+ support list/ranges/regex for version and define several
supported API a client could choose between
- code to share for I18n:
def prefered_language(available_languages)
lang_chooser = HTTPHeaders::AcceptLanguage.parse(self.preferredLanguage)
return nil if lang_chooser.nil?
ordered_list = lang_chooser.reduce(available_languages)
ordered_list.empty? ? nil : ordered_list.first.range
end
- leaf node feature: declare a node as leaf and pass all other node elements as parameters
Clerk:
- API:
/Command/* -> execute a command, each part being a node element:
+ needs leaf node feature
+ how to handle refs? (like in "DO SOMETHING FROM @1 TO @2")
preserving refs and giving objects via a parameter Hash seems a good solution
is the leaf node feature really needed?
/Batch(<script>, <parameters>) -> store a batch of commands (array of strings? array
of arrays of strings?) and parameters (hash), execute them, and later send a report
to the sender bot
- command description useful for shell-based IHM
Postman:
- process schema (without intermediate bot for command processing):
......
each queue is an internal channel, each processor is a task.
API (all bots):
- list manipulators:
+ /? -> search(<criterias>)
+ /+ -> create(<data>)
+ /- -> delete(<name>)
MapMaker:
- API:
/Services/DNS -> named info
......
/alter(<recipe>) -> recipe is a DSL-like list of commands
- diag functions for services
- distinguish presence of signed zone and which zone is really advertised by DNS
- detect available backends instead of having an enum in the config schema
Guard (auth bot):
- process for delegating auth (needs polish):
......
channels, so it is impossible to unsure a single call
- work on sending notif to peers through special channels
API:
- node def:
+ name
+ array of names
+ regex: matching names
+ nil: method_missing-like catchall
- client API:
+ exists?
+ call
+ [<name>]: sub-node
- define_interface:
support list/ranges/regex for version and define several
supported API a client could choose between
Channels:
- reorganize channel names
- enforce msg[:from] with 'task/<task-name>' in send_notification
bin/librarian
# to allow in-place run for test
$: << File.join(File.dirname(__FILE__), "..", "lib")
require 'needle'
require 'cyborghood/cyborg'
bin/mapmaker
# to allow in-place run for test
$: << File.join(File.dirname(__FILE__), "..", "lib")
require 'needle'
require 'cyborghood/cyborg'
bin/postman
# to allow in-place run for test
$: << File.join(File.dirname(__FILE__), "..", "lib")
#require 'socket'
require 'cyborghood'
require 'cyborghood/imap'
require 'cyborghood/mail'
require 'cyborghood/mail_order'
require 'cyborghood/command_runner'
#Socket.gethostname
require 'cyborghood/cyborg'
require 'cyborghood-postman/mail'
require 'cyborghood-postman/mail_order'
module CyborgHood
# example for specific validation rules of the bot-specific config file
#class Config
# protected
# class PostmanValidator < CyborgHoodValidator
# def validate_hook_in(value, rule, path, msg_list)
# msg_list << "Youhou !!!"
# end
# end
#end
module PostmanHome
module PostmanLand
include I18nTranslation
bindtextdomain("cyborghood_postman", {:path => Config::L10N_DIR, :charset => "UTF-8"})
# not yet ready to be a real Cyborg
class Postman #< Cyborg
include I18nTranslation
class Postman < Cyborg
# load config before modules are included
Config.load(self.human_name)
def initialize
# load config
Config.load(self.human_name.downcase)
@config = Config.instance
# setup logs
unless @config.log.nil?
logger.output_level(@config.log.console_level) unless @config.log.console_level.nil?
unless @config.log.path.nil?
if File.directory? @config.log.path
logger.log_to_file(File.join(@config.log.path, "ch_#{self.class.human_name}.log"))
else
logger.fatal "Log path does not exist or is not a directory, exiting"
exit 1
end
end
end
include BotNet
@imap = IMAP.new(@config.imap, @config.imap.min_check_interval)
def setup
super
logger.info "Bot '#{self.human_name}' loaded"
define_interface "0.1~"
end
def run
logger.info "Bot starting"
@imap.check_mails do |msg|
include I18nTranslation
def start_work
self.services.imap.check_mails do |msg|
begin
process_message(msg)
# Postman is not yet ready to process messages using the botnet
#process_message(msg)
rescue CyberError => e
raise
rescue
......
true
end
end
logger.info "Bot terminating"
end
def ask_to_stop
logger.info "Bot was asked to stop..."
@imap.stop_mail_check
super do
self.services.imap.stop_mail_check
end
end
def ready_to_stop?
super and not self.services.imap.connected?
end
private
......
end
end
bot = CyborgHood::PostmanHome::Postman.new
reg = Needle::Registry.new
reg.define do |b|
b.require 'cyborghood', CyborgHood
b.require 'cyborghood-postman/land', CyborgHood::PostmanLand
b.bot { CyborgHood::PostmanLand::Postman.new(b.postman_land) }
end
bot = reg.bot
trap('INT') do
bot.ask_to_stop
lib/cyborghood-postman/imap.rb
#--
# CyborgHood, a distributed system management software.
# Copyright (c) 2009-2011 Marc Dequènes (Duck) <Duck@DuckCorp.org>
#
# This program is free software: you can redistribute it and/or modify
# it under the terms of the GNU General Public License as published by
# the Free Software Foundation, either version 3 of the License, or
# (at your option) any later version.
#
# This program is distributed in the hope that it will be useful,
# but WITHOUT ANY WARRANTY; without even the implied warranty of
# MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE. See the
# GNU General Public License for more details.
#
# You should have received a copy of the GNU General Public License
# along with this program. If not, see <http://www.gnu.org/licenses/>.
#++
require 'net/imap'
require 'thread'
# IMAP IDLE support
class Net::IMAP
def idle(&response_handler)
return if @idle_mode
@idle_response_handler = response_handler
synchronize do
@idle_tag = generate_tag
add_response_handler @idle_response_handler if @idle_response_handler
put_string "#{@idle_tag} IDLE#{CRLF}"
end
@idle_mode = true
end
def idle_done
return unless @idle_mode
synchronize do
remove_response_handler @idle_response_handler if @idle_response_handler
put_string "DONE#{CRLF}"
end
@idle_mode = false
begin
return get_tagged_response @idle_tag
rescue
end
end
def idle?
@idle_mode || false
end
end
module CyborgHood
class IMAP
def initialize(params, min_check_interval)
@params = params
@min_check_interval = min_check_interval
@config = Config.instance
@imap = nil
@is_loggued = false
@available_mails = 0
@available_mails_mutex = Mutex.new
if @config.debug.flags.include?('debug_imapverbose')
Net::IMAP.debug = true
end
end
class IMAPMessage
def initialize(imap, message_id)
@imap = imap
@message_id = message_id
@config = Config.instance
@content = nil
@stop_mail_check = false
end
def content
return @content unless @content.nil?
data = @imap.fetch(@message_id, ["RFC822"])[0]
@content = data.attr["RFC822"]
end
def delete
if not @config.debug.flags.include?('debug_nomaildeletion')
@imap.store(@message_id, "+FLAGS", [:Deleted])
end
end
end
def capabilities
return @imap_capab if @imap_capab
@imap_capab = @imap.capability
end
def connect
return true if @imap
# using SSL because TLS does not work in the NET::IMAP library in Ruby 1.8
# (but available in 1.9)
ssl = (not @params.ca_cert.nil?)
port = @params.port || (ssl ? 993 : 143)
check_ca = ssl
logger.debug "Connecting to the IMAP server..."
begin
@imap = Net::IMAP.new(@params.host, port, ssl, @params.ca_cert, check_ca)
rescue SocketError
logger.warn "Could not connect to the IMAP server"
return false
end
logger.debug "Connected (IMAP Capabilities: " + self.capabilities.join(", ") + ")"
#p @imap.getquotaroot("INBOX")
@stop_mail_check = false
true
end
def connected?
not @imap.nil?
end
def authenticate
raise CyberError.new(:unrecoverable, "fixme", "Trying to authenticate to the IMAP server, but we are not connected yet") unless @imap
logger.debug "Authenticating to the IMAP server..."
begin
@imap.authenticate('LOGIN', @params.login, @params.passwd)
rescue Net::IMAP::NoResponseError
logger.warn "Could not authenticate to the IMAP server"
disconnect
return false
end
logger.debug "Authenticated as '#{@params.login}'"
@is_loggued = true
true
end
def listen_to_events
@imap.add_response_handler do |resp|
logger.debug "IMAP event: " + resp.inspect if @config.debug.flags.include?('debug_imapverbose')
if resp.kind_of?(Net::IMAP::UntaggedResponse) and resp.name == "EXISTS"
@available_mails_mutex.synchronize do
@available_mails = resp.data.to_i
logger.debug "*** Received new mails (#{@available_mails})" if @available_mails > 0
end
end
end
true
end
def check_mails_once(&message_handler)
connect &&
authenticate &&
listen_to_events &&
wait_and_read_mails(&message_handler)
rescue Net::IMAP::NoResponseError => e
logger.warn "IMAP error: " + e.message.strip
rescue Net::IMAP::Error => e
logger.error "IMAP error: " + e.message.strip
ensure
begin
disconnect
rescue
end
end
def remaining_mails
nb = nil
@available_mails_mutex.synchronize do
nb = @available_mails
end
nb
end
def waiting_mails?
remaining_mails > 0
end
def wait_and_read_mails(&message_handler)
until @stop_mail_check
begin
check_inbox(&message_handler)
# don't loop forever
return if @config.debug.flags.include?('debug_nomaildeletion')
end while waiting_mails?
if self.capabilities.include?("IDLE")
logger.debug "Waiting for new mails in idle mode"
@imap.idle
sleep(1) until @stop_mail_check or waiting_mails?
@imap.idle_done
end
end
end
def check_mails(&message_handler)
@stop_mail_check = false
until @stop_mail_check
begin
t = Time.now.to_i
r = check_mails_once(&message_handler)
t2 = Time.now.to_i
sleep_time = @min_check_interval - (t2 - t)
rescue SocketError, Errno::EPIPE, Errno::ECONNREFUSED, Errno::ETIMEDOUT, Errno::ENETUNREACH => e
logger.warn "Network error: " + e.message.strip
sleep_time = @min_check_interval
end
# wait before new check either if the IMAP server does not support IDLE mode
# or if an error occured
if sleep_time > 0 and not @stop_mail_check
logger.debug "Having a break before new check..."
t = Time.now.to_i
while Time.now.to_i - t < sleep_time and not @stop_mail_check
sleep(1)
end
end
end
end
def stop_mail_check
@stop_mail_check = true
end
def check_inbox(&message_handler)
logger.debug "Examining INBOX"
@imap.select('INBOX')
logger.debug "Starting mail check"
@imap.search(["ALL"], "UTF-8").each do |message_id|
break if @stop_mail_check
logger.debug "*** Fetched mail ##{message_id}"
unless message_handler.call IMAPMessage.new(@imap, message_id)
@stop_mail_check = true
break
end
end
logger.debug "Mail check finished"
@imap.expunge
end
def disconnect
return unless @imap
begin
@imap.logout if @is_loggued
logger.debug "Disconnecting from IMAP server..."
@imap.disconnect
logger.debug "Disconnected from IMAP server"
rescue IOError
logger.debug "Already diconnected from IMAP server"
ensure
@imap = nil
@is_loggued = false
@imap_capab = nil
end
end
end
end
lib/cyborghood-postman/land.rb
#--
# CyborgHood, a distributed system management software.
# Copyright (c) 2009-2011 Marc Dequènes (Duck) <Duck@DuckCorp.org>
#
# This program is free software: you can redistribute it and/or modify
# it under the terms of the GNU General Public License as published by
# the Free Software Foundation, either version 3 of the License, or
# (at your option) any later version.
#
# This program is distributed in the hope that it will be useful,
# but WITHOUT ANY WARRANTY; without even the implied warranty of
# MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE. See the
# GNU General Public License for more details.
#
# You should have received a copy of the GNU General Public License
# along with this program. If not, see <http://www.gnu.org/licenses/>.
#++
# ensure we can find the needed programs (should be handled somewhere else)
ENV['PATH'] = (ENV['PATH'].split(":") + ["/sbin", "/usr/sbin", "/usr/local/sbin"]).uniq.join(":")
module CyborgHood
module PostmanLand
def register_services(container)
container.namespace_define(:postman_land) do |b|
b.imap do
require 'cyborghood-postman/imap'
IMAP.new(b.config.imap, b.config.imap.min_check_interval)
end
end
end
module_function :register_services
end
end
lib/cyborghood-postman/mail.rb
#--
# CyborgHood, a distributed system management software.
# Copyright (c) 2009-2011 Marc Dequènes (Duck) <Duck@DuckCorp.org>
#
# This program is free software: you can redistribute it and/or modify
# it under the terms of the GNU General Public License as published by
# the Free Software Foundation, either version 3 of the License, or
# (at your option) any later version.
#
# This program is distributed in the hope that it will be useful,
# but WITHOUT ANY WARRANTY; without even the implied warranty of
# MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE. See the
# GNU General Public License for more details.
#
# You should have received a copy of the GNU General Public License
# along with this program. If not, see <http://www.gnu.org/licenses/>.
#++
require 'delegate'
require 'tmail'
require 'tmail_gpg'
require 'tmail_extra'
require 'net/smtp'
require 'fileutils'
require 'digest/md5'
# This class handles RFC3156 signed/encrypted messages, validates them, and extract content properly.
# It also implements a protection against replay attacks.
module CyborgHood
class MailReport
attr_reader :error, :warn_sender, :user, :message
def initialize(params = {})
@error = params[:error]
@warn_sender = params[:warn_sender]
@user = params[:user]
@message = params[:message]
@warn_sender = false
end
def ok?
@error.nil? and @user and @message
end
def warn_sender
# send reply message if user is identified or in selected situations
# (@warn_sender acts as an override)
@warn_sender || @user
end
end
class Mail < Delegator
include I18nTranslation
attr_accessor :user, :signature_timestamp
def initialize(msg = nil)
@config = Config.instance
if msg.nil?
@mail = TMail::Mail.new
set_custom_headers
elsif msg.is_a? TMail::Mail
@mail = msg
else
@mail = TMail::Mail.parse(msg)
end
end
def self.blank
m = self.new
m.clear
m
end
def __getobj__
@mail
end
def process
if is_marked?
if @config.debug.flags.include?('debug_ignorereplay')
logger.warn "Debug: ignoring replay"
else
return MailReport.new(:error => _("Replay detected."))
end
end
return process_signed() if is_pgp_signed?
return process_encrypted() if is_pgp_encrypted?
MailReport.new(:error => _("Mail not RFC3156 compliant."))
end
def create_reply
tmail_reply = @mail.create_reply
tmail_reply.from_addrs = TMail::Address.parse(@config.mail.from_address || self.to.first)
tmail_reply.to_addrs = (@mail.reply_to_addrs_pretty || @mail.from_addrs_pretty).collect do |a|
TMail::Address.parse(@mail.class.header_value_smtp(a))
end
reply = self.class.new(tmail_reply.to_s)
reply.set_custom_headers
reply.user = @user
reply
end
def create_simple_reject_reply(msg)
reply = create_reply()
reply.set_content_type("text", "plain", {'charset' => "utf-8"})
reply.set_disposition("inline")
reply.quoted_printable_body = msg + self.default_body_signature
reply.sign
reply.crypt(reply.user.keyFingerPrint) unless reply.user.nil?
reply
end
def to_multipart!
self.set_content_type("multipart", "mixed", {'boundary' => TMail.new_boundary})
self.transfer_encoding = "7bit"
self.quoted_body = "This mail is in MIME format.\n"
self.parts.clear
end
def set_custom_headers
@mail['Organization'] = @config.mail.organization
end
def check_headers
@mail.header.keys.each do |h|
@mail[h] = @mail.class.quote_address_if_necessary(@mail[h].to_s, "utf-8")
end
end
def default_body_signature
s = "\n" +
"-- \n" +
"#{CyborgHood::PRODUCT} v#{CyborgHood::VERSION}\n"
s += _("Contact eMail:").to_s + " \"#{@config.contact.name}\" <#{@config.contact.email}>\n" if @config.contact.email
s += _("Contact URL:").to_s + " #{@config.contact.url}\n" if @config.contact.url
s
end
def deliver
check_headers
smtp_server = @config.mail.smtp_server
smtp_port = @config.mail.smtp_port
smtp_from = @mail.from_addrs.collect{|a| a.address}.join(", ")
smtp_to = @mail.to_addrs.collect{|a| a.address}
Net::SMTP.start(smtp_server, smtp_port) do |smtp|
smtp.send_message(@mail.to_s, smtp_from, smtp_to)
end
end
def to_s
@mail.to_s
end
def crypt(fingerprint = nil)
@mail = @mail.create_encrypted(fingerprint || @user.keyFingerPrint)
end
def sign
# we don't check the uid, as there is only one signer
@mail = @mail.create_signed(@config.mail.key_id) do |uid_hint, passphrase_info, prev_was_bad|
@config.mail.key_passphrase
end
end
def sign_and_crypt(fingerprint = nil)
# not using sign_and_crypt(), to avoid repeating code
sign()
crypt(fingerprint || @user.keyFingerPrint)
end
private
def process_signed
sigs_check = verify_pgp_signature()
return MailReport.new(:error => _("Mail not formatted correctly (signed part).")) if sigs_check.nil? or sigs_check.size != 1
sig_check = sigs_check.first
return MailReport.new(:error => _("Mail content tampered or badly signed: %{sig_err}", :sig_err => sig_check.to_s)) unless sig_check.status == 0
logger.info "Mail content was properly signed by key #{sig_check.fingerprint}"
user = Person.find_by_fingerprint(sig_check.fingerprint)
return MailReport.new(:error => _("Mail is from an unknown person."), :warn_sender => true) if user.nil?
logger.info "Mail is from user #{user.uid} (#{user.cn})"
@user = user
drift = Time.new.to_i - sig_check.timestamp.to_i
logger.debug "Signature drift time: #{drift}"
unless drift.abs < @config.mail.max_drift_time
if drift > 0
return MailReport.new(:error => _("The signature was made too long ago (check your system clock). Rejected message to avoid replay attacks."), :user => user)
else
# mark message to prevent later replay of the message
mark_processed(sig_check.timestamp)
return MailReport.new(:error => _("The signature was made in the future (check your system clock). Rejected message to avoid replay attacks."), :user => user)
end
end
logger.debug "Mail OK"
mark_processed(self.signature_timestamp)
signed_content = pgp_signed_part()
# create a fake mail and chain parsing operations
plain_mail = self.class.new(signed_content)
plain_mail.user = @user
plain_mail.signature_timestamp = sig_check.timestamp
# propagate message_id to be able to mark messages (replay protection)
plain_mail.message_id = @mail.message_id
MailReport.new(:user => user, :message => plain_mail)
end
def process_encrypted
catch :notforme do
begin
# block is not passed to delegate (limitation ?)
clear_message = @mail.pgp_decrypt do |uid_hint, passphrase_info, prev_was_bad|
logger.info "Mail crypted for #{uid_hint}"
# check if requesting passphrase for the expected key
key_id = passphrase_info.split(" ")[1]
throw :notforme if key_id != @config.mail.key_id[-(key_id.size)..-1]
# sending key
@config.mail.key_passphrase
end
# create a fake mail and chain parsing operations
clear_mail = self.class.new(clear_message)
clear_mail.user = @user
# propagate message_id to be able to mark messages (replay protection)
clear_mail.message_id = @mail.message_id
decrypted_mail = clear_mail.process
# reverse propagate user information (convenience)
@user = decrypted_mail.user
return decrypted_mail
rescue GPGME::Error, NotImplementedError => e
raise CyberError.new(:unrecoverable, "protocol/mail", e.message)
end
end
MailReport.new(:error => _("Mail not formatted correctly (encrypted part)."))
end
def mark_dir
File.join(Config::VAR_DIR, "marks")
end
def mark_filename
File.join(mark_dir(), Digest::MD5.hexdigest(self.message_id))
end
def mark_processed(time)
FileUtils.mkdir_p(mark_dir())
File.open(mark_filename(), "w") do |fp|
fp.write self.message_id
end
File.utime(Time.now.to_i, time.to_i, mark_filename())
end
def is_marked?
File.exists?(mark_filename())
end
end
end
lib/cyborghood-postman/mail_order.rb
#--
# CyborgHood, a distributed system management software.
# Copyright (c) 2009-2011 Marc Dequènes (Duck) <Duck@DuckCorp.org>
#
# This program is free software: you can redistribute it and/or modify
# it under the terms of the GNU General Public License as published by
# the Free Software Foundation, either version 3 of the License, or
# (at your option) any later version.
#
# This program is distributed in the hope that it will be useful,
# but WITHOUT ANY WARRANTY; without even the implied warranty of
# MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE. See the
# GNU General Public License for more details.
#
# You should have received a copy of the GNU General Public License
# along with this program. If not, see <http://www.gnu.org/licenses/>.
#++
require 'cyborghood/order'
module CyborgHood
class MailOrderParser < OrderParser
def parse(message)
logger.debug "Parsing Mail"
# analyse mail parts
order_txt = nil
if message.multipart?
if message.parts[0].content_type == "text/plain"
order_txt = message.parts[0].body_normalized
i = -1
message.parts.each do |part|
i += 1
next if i == 0
@shared_parameters[i] = CommandParameter.new(part.body_normalized, part.content_type)
filename = part.header['content-disposition'].params['filename'] || part.header['content-type'].params['name']
@shared_parameters[filename] = CommandParameterRef.new(i) if filename
end
end
else
order_txt = message.body_normalized if message.content_type == "text/plain"
end
return Order.new(:error => _("Mail does not contain a proper text part for commands."), :user => @user) if order_txt.nil?
# find command lines
command_lines = order_txt.split("\n")
# remove message signature
signature_pos = command_lines.index("-- ")
command_lines.slice!(signature_pos..-1) if signature_pos
# generic parsing
super(command_lines) do |word, errors|
if word =~ /^@([a-zA-Z0-9._-]+)$/
ref = $1
ref = ref.is_numeric? ? ref.to_i : ref
d_ref, d_param = deref_param(ref)
if d_ref.nil?
errors << _("Attachment '%{ref}' not found.", :ref => ref)
d_param = nil
else
@used_refs << d_ref
end
d_param
else
CommandParameter.new(word, 'text/plain')
end
end
end
end
end
lib/cyborghood/cyborg.rb
def ask_to_stop
logger.info "Bot was asked to stop..."
yield if block_given?
try_stop
end
lib/cyborghood/imap.rb
#--
# CyborgHood, a distributed system management software.
# Copyright (c) 2009-2011 Marc Dequènes (Duck) <Duck@DuckCorp.org>
#
# This program is free software: you can redistribute it and/or modify
# it under the terms of the GNU General Public License as published by
# the Free Software Foundation, either version 3 of the License, or
# (at your option) any later version.
#
# This program is distributed in the hope that it will be useful,
# but WITHOUT ANY WARRANTY; without even the implied warranty of
# MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE. See the
# GNU General Public License for more details.
#
# You should have received a copy of the GNU General Public License
# along with this program. If not, see <http://www.gnu.org/licenses/>.
#++
require 'net/imap'
require 'thread'
# IMAP IDLE support
class Net::IMAP
def idle(&response_handler)
return if @idle_mode
@idle_response_handler = response_handler
synchronize do
@idle_tag = generate_tag
add_response_handler @idle_response_handler if @idle_response_handler
put_string "#{@idle_tag} IDLE#{CRLF}"
end
@idle_mode = true
end
def idle_done
return unless @idle_mode
synchronize do
remove_response_handler @idle_response_handler if @idle_response_handler
put_string "DONE#{CRLF}"
end
@idle_mode = false
begin
return get_tagged_response @idle_tag
rescue
end
end
def idle?
@idle_mode || false
end
end
module CyborgHood
class IMAP
def initialize(params, min_check_interval)
@params = params
@min_check_interval = min_check_interval
@config = Config.instance
@imap = nil
@is_loggued = false
@available_mails = 0
@available_mails_mutex = Mutex.new
if @config.debug.flags.include?('debug_imapverbose')
Net::IMAP.debug = true
end
end
class IMAPMessage
def initialize(imap, message_id)
@imap = imap
@message_id = message_id
@config = Config.instance
@content = nil
@stop_mail_check = false
end
def content
return @content unless @content.nil?
data = @imap.fetch(@message_id, ["RFC822"])[0]
@content = data.attr["RFC822"]
end
def delete
if not @config.debug.flags.include?('debug_nomaildeletion')
@imap.store(@message_id, "+FLAGS", [:Deleted])
end
end
end
def capabilities
return @imap_capab if @imap_capab
@imap_capab = @imap.capability
end
def connect
return true if @imap
# using SSL because TLS does not work in the NET::IMAP library in Ruby 1.8
# (but available in 1.9)
ssl = (not @params.ca_cert.nil?)
port = @params.port || (ssl ? 993 : 143)
check_ca = ssl
logger.debug "Connecting to the IMAP server..."
begin
@imap = Net::IMAP.new(@params.host, port, ssl, @params.ca_cert, check_ca)
rescue SocketError
logger.warn "Could not connect to the IMAP server"
return false
end
logger.debug "Connected (IMAP Capabilities: " + self.capabilities.join(", ") + ")"
#p @imap.getquotaroot("INBOX")
@stop_mail_check = false
true
end
def authenticate
raise CyberError.new(:unrecoverable, "fixme", "Trying to authenticate to the IMAP server, but we are not connected yet") unless @imap
logger.debug "Authenticating to the IMAP server..."
begin
@imap.authenticate('LOGIN', @params.login, @params.passwd)
rescue Net::IMAP::NoResponseError
logger.warn "Could not authenticate to the IMAP server"
disconnect
return false
end
logger.debug "Authenticated as '#{@params.login}'"
@is_loggued = true
true
end
def listen_to_events
@imap.add_response_handler do |resp|
logger.debug "IMAP event: " + resp.inspect if @config.debug.flags.include?('debug_imapverbose')
if resp.kind_of?(Net::IMAP::UntaggedResponse) and resp.name == "EXISTS"
@available_mails_mutex.synchronize do
@available_mails = resp.data.to_i
logger.debug "*** Received new mails (#{@available_mails})" if @available_mails > 0
end
end
end
true
end
def check_mails_once(&message_handler)
connect &&
authenticate &&
listen_to_events &&
wait_and_read_mails(&message_handler)
rescue Net::IMAP::NoResponseError => e
logger.warn "IMAP error: " + e.message.strip
rescue Net::IMAP::Error => e
logger.error "IMAP error: " + e.message.strip
ensure
begin
disconnect
rescue
end
end
def remaining_mails
nb = nil
@available_mails_mutex.synchronize do
nb = @available_mails
end
nb
end
def waiting_mails?
remaining_mails > 0
end
def wait_and_read_mails(&message_handler)
until @stop_mail_check
begin
check_inbox(&message_handler)
# don't loop forever
return if @config.debug.flags.include?('debug_nomaildeletion')
end while waiting_mails?
if self.capabilities.include?("IDLE")
logger.debug "Waiting for new mails in idle mode"
@imap.idle
sleep(1) until @stop_mail_check or waiting_mails?
@imap.idle_done
end
end
end
def check_mails(&message_handler)
@stop_mail_check = false
until @stop_mail_check
begin
t = Time.now.to_i
r = check_mails_once(&message_handler)
t2 = Time.now.to_i
sleep_time = @min_check_interval - (t2 - t)
rescue SocketError, Errno::EPIPE, Errno::ECONNREFUSED, Errno::ETIMEDOUT, Errno::ENETUNREACH => e
logger.warn "Network error: " + e.message.strip
sleep_time = @min_check_interval
end
# wait before new check either if the IMAP server does not support IDLE mode
# or if an error occured
if sleep_time > 0 and not @stop_mail_check
logger.debug "Having a break before new check..."
t = Time.now.to_i
while Time.now.to_i - t < sleep_time and not @stop_mail_check
sleep(1)
end
end
end
end
def stop_mail_check
@stop_mail_check = true
end
def check_inbox(&message_handler)
logger.debug "Examining INBOX"
@imap.select('INBOX')
logger.debug "Starting mail check"
@imap.search(["ALL"], "UTF-8").each do |message_id|
break if @stop_mail_check
logger.debug "*** Fetched mail ##{message_id}"
unless message_handler.call IMAPMessage.new(@imap, message_id)
@stop_mail_check = true
break
end
end
logger.debug "Mail check finished"
@imap.expunge
end
def disconnect
return unless @imap
begin
@imap.logout if @is_loggued
logger.debug "Disconnecting from IMAP server..."
@imap.disconnect
logger.debug "Disconnected from IMAP server"
rescue IOError
logger.debug "Already diconnected from IMAP server"
ensure
@imap = nil
@is_loggued = false
@imap_capab = nil
end
end
end
end
lib/cyborghood/mail.rb
#--
# CyborgHood, a distributed system management software.
# Copyright (c) 2009-2011 Marc Dequènes (Duck) <Duck@DuckCorp.org>
#
# This program is free software: you can redistribute it and/or modify
# it under the terms of the GNU General Public License as published by
# the Free Software Foundation, either version 3 of the License, or
# (at your option) any later version.
#
# This program is distributed in the hope that it will be useful,
# but WITHOUT ANY WARRANTY; without even the implied warranty of
# MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE. See the
# GNU General Public License for more details.
#
# You should have received a copy of the GNU General Public License
# along with this program. If not, see <http://www.gnu.org/licenses/>.
#++
require 'delegate'
require 'tmail'
require 'tmail_gpg'
require 'tmail_extra'
require 'net/smtp'
require 'fileutils'
require 'digest/md5'
# This class handles RFC3156 signed/encrypted messages, validates them, and extract content properly.
# It also implements a protection against replay attacks.
module CyborgHood
class MailReport
attr_reader :error, :warn_sender, :user, :message
def initialize(params = {})
@error = params[:error]
@warn_sender = params[:warn_sender]
@user = params[:user]
@message = params[:message]
@warn_sender = false
end
def ok?
@error.nil? and @user and @message
end
def warn_sender
# send reply message if user is identified or in selected situations
# (@warn_sender acts as an override)
@warn_sender || @user
end
end
class Mail < Delegator
include I18nTranslation
attr_accessor :user, :signature_timestamp
def initialize(msg = nil)
@config = Config.instance
if msg.nil?
@mail = TMail::Mail.new
set_custom_headers
elsif msg.is_a? TMail::Mail
@mail = msg
else
@mail = TMail::Mail.parse(msg)
end
end
def self.blank
m = self.new
m.clear
m
end
def __getobj__
@mail
end
def process
if is_marked?
if @config.debug.flags.include?('debug_ignorereplay')
logger.warn "Debug: ignoring replay"
else
return MailReport.new(:error => _("Replay detected."))
end
end
return process_signed() if is_pgp_signed?
return process_encrypted() if is_pgp_encrypted?
MailReport.new(:error => _("Mail not RFC3156 compliant."))
end
def create_reply
tmail_reply = @mail.create_reply
tmail_reply.from_addrs = TMail::Address.parse(@config.mail.from_address || self.to.first)
tmail_reply.to_addrs = (@mail.reply_to_addrs_pretty || @mail.from_addrs_pretty).collect do |a|
TMail::Address.parse(@mail.class.header_value_smtp(a))
end
reply = self.class.new(tmail_reply.to_s)
reply.set_custom_headers
reply.user = @user
reply
end
def create_simple_reject_reply(msg)
reply = create_reply()
reply.set_content_type("text", "plain", {'charset' => "utf-8"})
reply.set_disposition("inline")
reply.quoted_printable_body = msg + self.default_body_signature
reply.sign
reply.crypt(reply.user.keyFingerPrint) unless reply.user.nil?
reply
end
def to_multipart!
self.set_content_type("multipart", "mixed", {'boundary' => TMail.new_boundary})
self.transfer_encoding = "7bit"
self.quoted_body = "This mail is in MIME format.\n"
self.parts.clear
end
def set_custom_headers
@mail['Organization'] = @config.mail.organization
end
def check_headers
@mail.header.keys.each do |h|
@mail[h] = @mail.class.quote_address_if_necessary(@mail[h].to_s, "utf-8")
end
end
def default_body_signature
s = "\n" +
"-- \n" +
"#{CyborgHood::PRODUCT} v#{CyborgHood::VERSION}\n"
s += _("Contact eMail:").to_s + " \"#{@config.contact.name}\" <#{@config.contact.email}>\n" if @config.contact.email
s += _("Contact URL:").to_s + " #{@config.contact.url}\n" if @config.contact.url
s
end
def deliver
check_headers
smtp_server = @config.mail.smtp_server
smtp_port = @config.mail.smtp_port
smtp_from = @mail.from_addrs.collect{|a| a.address}.join(", ")
smtp_to = @mail.to_addrs.collect{|a| a.address}
Net::SMTP.start(smtp_server, smtp_port) do |smtp|
smtp.send_message(@mail.to_s, smtp_from, smtp_to)
end
end
def to_s
@mail.to_s
end
def crypt(fingerprint = nil)
@mail = @mail.create_encrypted(fingerprint || @user.keyFingerPrint)
end
def sign
# we don't check the uid, as there is only one signer
@mail = @mail.create_signed(@config.mail.key_id) do |uid_hint, passphrase_info, prev_was_bad|
@config.mail.key_passphrase
end
end
def sign_and_crypt(fingerprint = nil)
# not using sign_and_crypt(), to avoid repeating code
sign()
crypt(fingerprint || @user.keyFingerPrint)
end
private
def process_signed
sigs_check = verify_pgp_signature()
return MailReport.new(:error => _("Mail not formatted correctly (signed part).")) if sigs_check.nil? or sigs_check.size != 1
sig_check = sigs_check.first
return MailReport.new(:error => _("Mail content tampered or badly signed: %{sig_err}", :sig_err => sig_check.to_s)) unless sig_check.status == 0
logger.info "Mail content was properly signed by key #{sig_check.fingerprint}"
user = Person.find_by_fingerprint(sig_check.fingerprint)
return MailReport.new(:error => _("Mail is from an unknown person."), :warn_sender => true) if user.nil?
logger.info "Mail is from user #{user.uid} (#{user.cn})"
@user = user
drift = Time.new.to_i - sig_check.timestamp.to_i
logger.debug "Signature drift time: #{drift}"
unless drift.abs < @config.mail.max_drift_time
if drift > 0
return MailReport.new(:error => _("The signature was made too long ago (check your system clock). Rejected message to avoid replay attacks."), :user => user)
else
# mark message to prevent later replay of the message
mark_processed(sig_check.timestamp)
return MailReport.new(:error => _("The signature was made in the future (check your system clock). Rejected message to avoid replay attacks."), :user => user)
end
end
logger.debug "Mail OK"
mark_processed(self.signature_timestamp)
signed_content = pgp_signed_part()
# create a fake mail and chain parsing operations
plain_mail = self.class.new(signed_content)
plain_mail.user = @user
plain_mail.signature_timestamp = sig_check.timestamp
# propagate message_id to be able to mark messages (replay protection)
plain_mail.message_id = @mail.message_id
MailReport.new(:user => user, :message => plain_mail)
end
def process_encrypted
catch :notforme do
begin
# block is not passed to delegate (limitation ?)
clear_message = @mail.pgp_decrypt do |uid_hint, passphrase_info, prev_was_bad|
logger.info "Mail crypted for #{uid_hint}"
# check if requesting passphrase for the expected key
key_id = passphrase_info.split(" ")[1]
throw :notforme if key_id != @config.mail.key_id[-(key_id.size)..-1]
# sending key
@config.mail.key_passphrase
end
# create a fake mail and chain parsing operations
clear_mail = self.class.new(clear_message)
clear_mail.user = @user
# propagate message_id to be able to mark messages (replay protection)
clear_mail.message_id = @mail.message_id
decrypted_mail = clear_mail.process
# reverse propagate user information (convenience)
@user = decrypted_mail.user
return decrypted_mail
rescue GPGME::Error, NotImplementedError => e
raise CyberError.new(:unrecoverable, "protocol/mail", e.message)
end
end
MailReport.new(:error => _("Mail not formatted correctly (encrypted part)."))
end
def mark_dir
File.join(Config::VAR_DIR, "marks")
end
def mark_filename
File.join(mark_dir(), Digest::MD5.hexdigest(self.message_id))
end
def mark_processed(time)
FileUtils.mkdir_p(mark_dir())
File.open(mark_filename(), "w") do |fp|
fp.write self.message_id
end
File.utime(Time.now.to_i, time.to_i, mark_filename())
end
def is_marked?
File.exists?(mark_filename())
end
end
end
lib/cyborghood/mail_order.rb
#--
# CyborgHood, a distributed system management software.
# Copyright (c) 2009-2011 Marc Dequènes (Duck) <Duck@DuckCorp.org>
#
# This program is free software: you can redistribute it and/or modify
# it under the terms of the GNU General Public License as published by
... This diff was truncated because it exceeds the maximum size that can be displayed.

Also available in: Unified diff