rTorrent: Erst-Seeding
Posted by hyper_ch on 04 May 2008 | Tagged as: CLI, hyper_ch
UPDATE
Das Erst-Seeding ist in der aktuellen SVN-Version (revision 1060) enthalten. Das Tutorial hier ist also veraltet.
Erst-Seeding
Kreiert man einen neuen Torrent, der doch relativ gross ist, und lädt man ihn hoch, dann möchte ich zumindesta aus Effizienzegründen zuerst alle Chunkgs einmal seeden bevor rTorrent dann entsprechend den Anfragen der anderen Clients Chunks hochlädt. Standardmässig kann rTorrent das nicht, aber Josef (von irc.worldforge.net #libtorrent) hat dazu einen Patch geschrieben.
1. Erst-Seeding Patch herunterladen
Der aktuelle Patch kann von http://ovh.ttdpatch.net/~jdrexler/rt/initial-seed.diff heruntergeladen werden. Da ich nicht weiss, wie lange da zur Verfügung steht, werde ich die aktuelle Version noch anhängen. Der Patch muss im trunk Verzeichnis von rtorrent gespeichert werden.
2. Source Code patchen
Das Patchen des Source Code geht eigentlich ganz einfach. Einfach ins Trunk Verzeichnis gehen und dort folgenden Befehl ausführen:
patch -p0 < initial-seed.diff
3. Neu kompilieren
Gemäss dem Tutorial hiermuss einfach libtorrent und rtorrent nochmals kompiliert und installiert werden. Das Ausführen von autogen und configure ist nicht notwendig.
4. Anwenden
Wurde rTorrent gepatcht und installiert, dann kann das Erst-Seeding folgendermassen vorgenommen werden.
- Den gewünschten Torrent stoppen mit ctrl-k
- Dann als Erst-Seeding markieren mit ctrl-b
- Torrent wieder starten mit ctrl-s
In der Torrent-Info müsste dann als Connection Type nun auch “initial_seed” stehen.
Gemäss Josef wird auch noch empfohlen, die max/min Geschwindigkeiten und Slots anzupassen.
5. Erst-Seeding aufheben
Um das Erst-Seeding aufzuheben, also bevor der Schwar min. 2 komplette Kopien anzeigt, muss der Torrent
- geschlossen werden: ctrl-k
- dann mit ctrl-x folgender Befehl eingegeben werden: d.set_connection_seed=seed
- und dann den Torrent wieder starten: ctrl-s
5. Aktuelle Erst-Seed
Für Revision 1057 (möglicherweise dann auch spätere):
# =================================================================
#
# Using initial seeding
#
# Initial seeding is useful only when you are the very first seed on
# a torrent, and helps it get seeded faster by uploading each chunk
# only once. See http://bittorrent.org/beps/bep_0016.html for
# technical details. It is less useful if your upload bandwidth is
# much higher than the average peer's upload however.
#
# To enable it on a download, first make sure the download is complete
# and that it is stopped, then press Ctrl-B or send it the command
# d.set_connection_seed=initial_seed
# It will probably not show the proper connection type in the download
# info until you actually start it afterwards. If it shows "leech" or
# "seed" even after you start it, it is NOT initial seeding. This may
# be because it found other seeds in the swarm or all chunks are
# already sufficiently distributed.
#
# In addition to the algorithm described in BEP 0016, rtorrent's
# initial seeding includes a fast-startup mode to allow full upload
# bandwidth even when there are very few peers. Normally initial seeding
# waits for chunks to appear elsewhere in the swarm before allowing a
# peer to download more. But when the total number of interested peers
# is less than 90% of the max uploads for the download, the peer may
# download more right away since we don't have any better use for
# the upload bandwidth of the download.
#
# This means that to tune initial seeding, set the download's max uploads
# to about your global upload slot limit, and the download's max peers to
# about 50%-100% higher. For example, if you have 20 global slots, set the
# download's max uploads to 20 as well, and the max peers to 30-40.
# This assumes the download is the only active download. Adjust accordingly
# if there are others.
#
# When initial seeding completes, all connections are closed, and the
# non-seeders re-connected as normal seeding connection. This is another
# reason why initial seeding should not be enabled unless you really are
# the very first seed. The connection type in the download info will also
# change to "seed" so you may know that initial seeding is complete.
#
# TESTING
#
# This patch is experimental and has not been test much "in the wild". Please
# do test it, at worst it will probably stall near the end of initial seeding.
# Monitor the chunk statistics to see if it is getting close to being completely
# seeded, if the upload speed drops sharply at that point and does not recover
# after several minutes, please stop the download and start it again (still in
# initial seeding mode). This should let it recover and complete initial seeding
# normally. If it still does not recover, stop it and manually set it to normal
# seeding with Ctrl-X "d.set_connection_seed=seed" and start it again.
#
# Please report all your finding to josef on IRC (#libtorrent@irc.worldforge.org)
# or email josef.at.ttdpatch.net so I know what (if anything) needs improving.
#
# =================================================================
Index: rtorrent/src/core/download_list.cc
===================================================================
--- rtorrent/src/core/download_list.cc (revision 1045)
+++ rtorrent/src/core/download_list.cc (working copy)
@@ -454,6 +454,10 @@
rpc::call_command("d.set_state_changed", cachedTime.seconds(), rpc::make_target(download));
rpc::call_command("d.set_state_counter", rpc::call_command_value("d.get_state_counter", rpc::make_target(download)), rpc::make_target(download));
+ // If initial seeding is complete, don't try it again when restarting.
+ if (download->is_done())
+ rpc::call_command("d.set_connection_seed", rpc::call_command_void("d.get_connection_current", rpc::make_target(download)), rpc::make_target(download));
+
// Save the state after all the slots, etc have been called so we
// include the modifications they may make.
//control->core()->download_store()->save(download);
Index: rtorrent/src/core/download_factory.cc
===================================================================
--- rtorrent/src/core/download_factory.cc (revision 1045)
+++ rtorrent/src/core/download_factory.cc (working copy)
@@ -195,10 +195,6 @@
if (!rtorrent->has_key_string("custom4")) rtorrent->insert_key("custom4", std::string());
if (!rtorrent->has_key_string("custom5")) rtorrent->insert_key("custom5", std::string());
- // Move to 'rtorrent'.
- rpc::call_command("d.set_connection_leech", m_variables["connection_leech"], rpc::make_target(download));
- rpc::call_command("d.set_connection_seed", m_variables["connection_seed"], rpc::make_target(download));
-
rpc::call_command("d.set_uploads_max", rpc::call_command_void("get_max_uploads"), rpc::make_target(download));
rpc::call_command("d.set_peers_min", rpc::call_command_void("get_min_peers"), rpc::make_target(download));
rpc::call_command("d.set_peers_max", rpc::call_command_void("get_max_peers"), rpc::make_target(download));
@@ -354,6 +350,9 @@
download->download()->file_list()->size_chunks()));
rtorrent->insert_preserve_copy("ignore_commands", (int64_t)0);
+
+ rtorrent->insert_preserve_type("connection_leech", m_variables["connection_leech"]);
+ rtorrent->insert_preserve_type("connection_seed", m_variables["connection_seed"]);
}
}
Index: rtorrent/src/command_download.cc
===================================================================
--- rtorrent/src/command_download.cc (revision 1045)
+++ rtorrent/src/command_download.cc (working copy)
@@ -169,6 +169,8 @@
connType = torrent::Download::CONNECTION_LEECH;
else if (name == "seed")
connType = torrent::Download::CONNECTION_SEED;
+ else if (name == "initial_seed")
+ connType = torrent::Download::CONNECTION_INITIAL_SEED;
else
throw torrent::input_error("Unknown peer connection type selected.");
@@ -192,6 +194,8 @@
return "leech";
case torrent::Download::CONNECTION_SEED:
return "seed";
+ case torrent::Download::CONNECTION_INITIAL_SEED:
+ return "initial_seed";
default:
return "unknown";
}
@@ -473,16 +477,13 @@
ADD_CD_VARIABLE_VALUE_PUBLIC("ignore_commands", "rtorrent", "ignore_commands");
ADD_CD_STRING_BI("connection_current", std::ptr_fun(&apply_d_connection_type), std::ptr_fun(&retrieve_d_connection_type));
+ ADD_CD_VARIABLE_STRING("connection_leech", "rtorrent", "connection_leech");
+ ADD_CD_VARIABLE_STRING("connection_seed", "rtorrent", "connection_seed");
// This command really needs to be improved, so we have proper
// logging support.
ADD_CD_STRING_BI("message", std::mem_fun(&core::Download::set_message), std::mem_fun(&core::Download::message));
- add_copy_to_download("get_connection_leech", "d.get_connection_leech");
- add_copy_to_download("set_connection_leech", "d.set_connection_leech");
- add_copy_to_download("get_connection_seed", "d.get_connection_seed");
- add_copy_to_download("set_connection_seed", "d.set_connection_seed");
-
ADD_CD_VALUE_MEM_BI("max_file_size", &core::Download::file_list, &torrent::FileList::set_max_file_size, &torrent::FileList::max_file_size);
ADD_CD_VALUE_MEM_BI("peers_min", &core::Download::connection_list, &torrent::ConnectionList::set_min_size, &torrent::ConnectionList::min_size);
Index: rtorrent/src/ui/element_download_list.cc
===================================================================
--- rtorrent/src/ui/element_download_list.cc (revision 1045)
+++ rtorrent/src/ui/element_download_list.cc (working copy)
@@ -76,6 +76,10 @@
"branch=d.get_ignore_commands=,"
"{d.set_ignore_commands=0, print=\"Torrent set to heed commands.\"},"
"{d.set_ignore_commands=1, print=\"Torrent set to ignore commands.\"}");
+ m_bindings['B'-'@']= sigc::bind(sigc::mem_fun(*this, &ElementDownloadList::receive_command),
+ "branch=d.is_active=,"
+ "{print=\"Cannot enable initial seeding on an active download.\"},"
+ "{d.set_connection_seed=initial_seed, print=\"Enabled initial seeding for the selected download.\"}");
m_bindings['U'] = sigc::bind(sigc::mem_fun(*this, &ElementDownloadList::receive_command), "d.delete_tied=; print=\"Cleared tied to file association for the selected download.\"");
Index: libtorrent/src/torrent/peer/peer_info.cc
===================================================================
--- libtorrent/src/torrent/peer/peer_info.cc (revision 1045)
+++ libtorrent/src/torrent/peer/peer_info.cc (working copy)
@@ -69,6 +69,9 @@
if (m_transferCounter != 0)
throw internal_error("PeerInfo::~PeerInfo() m_transferCounter != 0.");
+ if (is_blocked())
+ throw internal_error("PeerInfo::~PeerInfo() peer is blocked.");
+
delete rak::socket_address::cast_from(m_address);
}
Index: libtorrent/src/torrent/peer/peer_list.cc
===================================================================
--- libtorrent/src/torrent/peer/peer_list.cc (revision 1045)
+++ libtorrent/src/torrent/peer/peer_list.cc (working copy)
@@ -306,7 +306,8 @@
itr->second->transfer_counter() != 0 ||
itr->second->last_connection() >= timer ||
- (flags & cull_keep_interesting && itr->second->failed_counter() != 0)) {
+ (flags & cull_keep_interesting &&
+ (itr->second->failed_counter() != 0 || itr->second->is_blocked()))) {
itr++;
continue;
}
Index: libtorrent/src/torrent/peer/peer_info.h
===================================================================
--- libtorrent/src/torrent/peer/peer_info.h (revision 1045)
+++ libtorrent/src/torrent/peer/peer_info.h (working copy)
@@ -47,12 +47,15 @@
friend class ConnectionList;
friend class Handshake;
friend class HandshakeManager;
+ friend class InitialSeeding;
friend class PeerList;
friend class ProtocolExtension;
static const int flag_connected = (1 << 0);
static const int flag_incoming = (1 << 1);
static const int flag_handshake = (1 << 2);
+ static const int flag_blocked = (1 << 3); // For initial seeding.
+ static const int flag_restart = (1 << 4);
PeerInfo(const sockaddr* address);
~PeerInfo();
@@ -60,6 +63,8 @@
bool is_connected() const { return m_flags & flag_connected; }
bool is_incoming() const { return m_flags & flag_incoming; }
bool is_handshake() const { return m_flags & flag_handshake; }
+ bool is_blocked() const { return m_flags & flag_blocked; }
+ bool is_restart() const { return m_flags & flag_restart; }
int flags() const { return m_flags; }
Index: libtorrent/src/torrent/download.cc
===================================================================
--- libtorrent/src/torrent/download.cc (revision 1045)
+++ libtorrent/src/torrent/download.cc (working copy)
@@ -106,6 +106,11 @@
// file_list()->open(flags);
file_list()->open(flags & ~FileList::open_no_create);
+ if (m_ptr->connection_type() == CONNECTION_INITIAL_SEED) {
+ if (!m_ptr->main()->start_initial_seeding())
+ set_connection_type(CONNECTION_SEED);
+ }
+
m_ptr->main()->start();
m_ptr->main()->tracker_manager()->set_active(true);
@@ -501,6 +506,11 @@
case CONNECTION_SEED:
m_ptr->main()->connection_list()->slot_new_connection(&createPeerConnectionSeed);
break;
+ case CONNECTION_INITIAL_SEED:
+ if (is_active() && m_ptr->main()->initial_seeding() == NULL)
+ throw input_error("Can't switch to initial seeding: download is active.");
+ m_ptr->main()->connection_list()->slot_new_connection(&createPeerConnectionInitialSeed);
+ break;
default:
throw input_error("torrent::Download::set_connection_type(...) received an unknown type.");
};
Index: libtorrent/src/torrent/download.h
===================================================================
--- libtorrent/src/torrent/download.h (revision 1045)
+++ libtorrent/src/torrent/download.h (working copy)
@@ -180,7 +180,8 @@
typedef enum {
CONNECTION_LEECH,
- CONNECTION_SEED
+ CONNECTION_SEED,
+ CONNECTION_INITIAL_SEED,
} ConnectionType;
ConnectionType connection_type() const;
Index: libtorrent/src/protocol/initial_seed.h
===================================================================
--- libtorrent/src/protocol/initial_seed.h (revision 0)
+++ libtorrent/src/protocol/initial_seed.h (revision 0)
@@ -0,0 +1,85 @@
+// libTorrent - BitTorrent library
+// Copyright (C) 2005-2007, Jari Sundell
+//
+// This program is free software; you can redistribute it and/or modify
+// it under the terms of the GNU General Public License as published by
+// the Free Software Foundation; either version 2 of the License, or
+// (at your option) any later version.
+//
+// This program is distributed in the hope that it will be useful,
+// but WITHOUT ANY WARRANTY; without even the implied warranty of
+// MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE. See the
+// GNU General Public License for more details.
+//
+// You should have received a copy of the GNU General Public License
+// along with this program; if not, write to the Free Software
+// Foundation, Inc., 59 Temple Place, Suite 330, Boston, MA 02111-1307 USA
+//
+// In addition, as a special exception, the copyright holders give
+// permission to link the code of portions of this program with the
+// OpenSSL library under certain conditions as described in each
+// individual source file, and distribute linked combinations
+// including the two.
+//
+// You must obey the GNU General Public License in all respects for
+// all of the code used other than OpenSSL. If you modify file(s)
+// with this exception, you may extend this exception to your version
+// of the file(s), but you are not obligated to do so. If you do not
+// wish to do so, delete this exception statement from your version.
+// If you delete this exception statement from all source files in the
+// program, then also delete it here.
+//
+// Contact: Jari Sundell <jaris@ifi.uio.no>
+//
+// Skomakerveien 33
+// 3185 Skoppum, NORWAY
+
+#ifndef LIBTORRENT_PROTOCOL_INITIAL_SEED_H
+#define LIBTORRENT_PROTOCOL_INITIAL_SEED_H
+
+#include "download/download_main.h"
+
+namespace torrent {
+
+class InitialSeeding {
+public:
+ InitialSeeding(DownloadMain* download);
+ ~InitialSeeding();
+
+ static const uint32_t no_offer = ~uint32_t();
+
+ void new_peer(PeerConnectionBase* pcb);
+
+ // Chunk was seen distributed to a peer in the swarm.
+ void chunk_seen(uint32_t index, PeerConnectionBase* pcb);
+
+ // Returns chunk we may offer the peer or no_offer if none.
+ uint32_t chunk_offer(PeerConnectionBase* pcb, uint32_t indexDone);
+
+ // During the second stage (seeding rare chunks), return
+ // false if given chunk is already well-seeded now. True otherwise.
+ bool should_upload(uint32_t index);
+
+private:
+ static PeerInfo* const chunk_unsent; // Chunk never sent to anyone.
+ static PeerInfo* const chunk_unknown; // Peer has chunk, we don't know who we sent it to.
+ static PeerInfo* const chunk_done; // Chunk properly distributed by peer.
+
+ uint32_t find_next(bool secondary, PeerConnectionBase* pcb);
+
+ bool valid_peer(PeerInfo* peer);
+ void clear_peer(PeerInfo* peer);
+ void chunk_complete(uint32_t index, PeerConnectionBase* pcb);
+
+ void complete(PeerConnectionBase* pcb);
+ void unblock_all();
+
+ uint32_t m_nextChunk;
+ uint32_t m_chunksLeft;
+ DownloadMain* m_download;
+ PeerInfo** m_peerChunks;
+};
+
+}
+
+#endif
Index: libtorrent/src/protocol/peer_connection_base.h
===================================================================
--- libtorrent/src/protocol/peer_connection_base.h (revision 1045)
+++ libtorrent/src/protocol/peer_connection_base.h (working copy)
@@ -136,6 +136,10 @@
void cancel_transfer(BlockTransfer* transfer);
+ // Insert into the poll unless we're blocking for throttling etc.
+ void read_insert_poll_safe();
+ void write_insert_poll_safe();
+
protected:
static const uint32_t extension_must_encrypt = ~uint32_t();
@@ -176,10 +180,6 @@
bool send_pex_message();
- // Insert into the poll unless we're blocking for throttling etc.
- void read_insert_poll_safe();
- void write_insert_poll_safe();
-
DownloadMain* m_download;
ProtocolRead* m_down;
Index: libtorrent/src/protocol/peer_connection_leech.cc
===================================================================
--- libtorrent/src/protocol/peer_connection_leech.cc (revision 1045)
+++ libtorrent/src/protocol/peer_connection_leech.cc (working copy)
@@ -50,6 +50,7 @@
#include "torrent/peer/peer_info.h"
#include "extensions.h"
+#include "initial_seed.h"
#include "peer_connection_leech.h"
namespace torrent {
@@ -65,6 +66,16 @@
template<Download::ConnectionType type>
void
PeerConnection<type>::initialize_custom() {
+ if (type == Download::CONNECTION_INITIAL_SEED) {
+ if (m_download->initial_seeding() == NULL) {
+ // Can't throw close_connection or network_error here, we're still
+ // initializing. So close the socket and let that kill it later.
+ get_fd().close();
+ return;
+ }
+
+ m_download->initial_seeding()->new_peer(this);
+ }
// if (m_download->content()->chunks_completed() != 0) {
// m_up->write_bitfield(m_download->file_list()->bitfield()->size_bytes());
@@ -77,7 +88,7 @@
template<Download::ConnectionType type>
void
PeerConnection<type>::update_interested() {
- if (type == Download::CONNECTION_SEED)
+ if (type != Download::CONNECTION_LEECH)
return;
m_peerChunks.download_cache()->clear();
@@ -117,7 +128,7 @@
m_encryption.encrypt(old_end, m_up->buffer()->end() - old_end);
}
- if (type == Download::CONNECTION_SEED)
+ if (type != Download::CONNECTION_LEECH)
return true;
m_tryRequest = true;
@@ -192,7 +203,7 @@
switch (buf->read_8()) {
case ProtocolBase::CHOKE:
- if (type == Download::CONNECTION_SEED)
+ if (type != Download::CONNECTION_LEECH)
return true;
// Cancel before dequeueing so receive_download_choke knows if it
@@ -212,7 +223,7 @@
return true;
case ProtocolBase::UNCHOKE:
- if (type == Download::CONNECTION_SEED)
+ if (type != Download::CONNECTION_LEECH)
return true;
m_downUnchoked = true;
@@ -258,7 +269,7 @@
return true;
case ProtocolBase::PIECE:
- if (type == Download::CONNECTION_SEED)
+ if (type != Download::CONNECTION_LEECH)
throw communication_error("Received a piece but the connection is strictly for seeding.");
if (!m_down->can_read_piece_body())
@@ -381,7 +392,7 @@
}
case ProtocolRead::READ_PIECE:
- if (type == Download::CONNECTION_SEED)
+ if (type != Download::CONNECTION_LEECH)
return;
if (!download_queue()->is_downloading())
@@ -401,7 +412,7 @@
break;
case ProtocolRead::READ_SKIP_PIECE:
- if (type == Download::CONNECTION_SEED)
+ if (type != Download::CONNECTION_LEECH)
return;
if (download_queue()->transfer()->is_leader()) {
@@ -489,12 +500,12 @@
// e.g. BitTornado 0.7.14 and uTorrent 0.3.0, disconnect if a
// request has been received while uninterested. The problem arises
// as they send unchoke before receiving interested.
- if (type != Download::CONNECTION_SEED && m_sendInterested && m_up->can_write_interested()) {
+ if (type == Download::CONNECTION_LEECH && m_sendInterested && m_up->can_write_interested()) {
m_up->write_interested(m_downInterested);
m_sendInterested = false;
}
- if (type != Download::CONNECTION_SEED && m_tryRequest) {
+ if (type == Download::CONNECTION_LEECH && m_tryRequest) {
if (!(m_tryRequest = !should_request()) &&
!(m_tryRequest = try_request_pieces()) &&
@@ -508,7 +519,7 @@
DownloadMain::have_queue_type* haveQueue = m_download->have_queue();
- if (type != Download::CONNECTION_SEED &&
+ if (type == Download::CONNECTION_LEECH &&
!haveQueue->empty() &&
m_peerChunks.have_timer() <= haveQueue->front().first &&
m_up->can_write_have()) {
@@ -522,7 +533,10 @@
m_peerChunks.set_have_timer(last->first + 1);
}
- while (type != Download::CONNECTION_SEED && !m_peerChunks.cancel_queue()->empty() && m_up->can_write_cancel()) {
+ if (type == Download::CONNECTION_INITIAL_SEED && m_up->can_write_have())
+ offer_chunk();
+
+ while (type == Download::CONNECTION_LEECH && !m_peerChunks.cancel_queue()->empty() && m_up->can_write_cancel()) {
m_up->write_cancel(m_peerChunks.cancel_queue()->front());
m_peerChunks.cancel_queue()->pop_front();
}
@@ -533,7 +547,8 @@
} else if (!m_upChoke.choked() &&
!m_peerChunks.upload_queue()->empty() &&
- m_up->can_write_piece()) {
+ m_up->can_write_piece() &&
+ (type != Download::CONNECTION_INITIAL_SEED || should_upload())) {
write_prepare_piece();
}
@@ -638,14 +653,20 @@
m_download->chunk_statistics()->received_have_chunk(&m_peerChunks, index, m_download->file_list()->chunk_size());
+ if (type == Download::CONNECTION_INITIAL_SEED)
+ m_download->initial_seeding()->chunk_seen(index, this);
+
+ // Disconnect seeds when we are seeding (but not for initial seeding
+ // so that we keep accurate chunk statistics until that is done).
if (m_peerChunks.bitfield()->is_all_set()) {
- if (type == Download::CONNECTION_SEED || m_download->file_list()->is_done())
+ if (type == Download::CONNECTION_SEED ||
+ (type != Download::CONNECTION_INITIAL_SEED && m_download->file_list()->is_done()))
throw close_connection();
m_download->upload_choke_manager()->set_not_queued(this, &m_upChoke);
}
- if (type == Download::CONNECTION_SEED || m_download->file_list()->is_done())
+ if (type != Download::CONNECTION_LEECH || m_download->file_list()->is_done())
return;
if (is_down_interested()) {
@@ -676,8 +697,58 @@
}
}
+template<>
+void
+PeerConnection<Download::CONNECTION_INITIAL_SEED>::offer_chunk() {
+ // If bytes left to send in this chunk minus bytes about to be sent is zero,
+ // assume the peer will have got the chunk completely. In that case we may
+ // get another one to offer if not enough other peers are interested even
+ // if the peer would otherwise still be blocked.
+ uint32_t bytesLeft = m_data.bytesLeft;
+ if (!m_peerChunks.upload_queue()->empty() && m_peerChunks.upload_queue()->front().index() == m_data.lastIndex)
+ bytesLeft -= m_peerChunks.upload_queue()->front().length();
+
+ uint32_t index = m_download->initial_seeding()->chunk_offer(this, bytesLeft == 0 ? m_data.lastIndex : InitialSeeding::no_offer);
+
+ if (index == InitialSeeding::no_offer || index == m_data.lastIndex)
+ return;
+
+ m_up->write_have(index);
+ m_data.lastIndex = index;
+ m_data.bytesLeft = m_download->file_list()->chunk_index_size(index);
+}
+
+// For initial seeding, check if chunk is well seeded now, and if so
+// remove it from the queue to better use our bandwidth on rare chunks.
+// If queue ends up empty, choke peer to let it know that it shouldn't
+// wait for pieces to be sent.
+template<>
+bool
+PeerConnection<Download::CONNECTION_INITIAL_SEED>::should_upload() {
+ while (!m_peerChunks.upload_queue()->empty() &&
+ !m_download->initial_seeding()->should_upload(m_peerChunks.upload_queue()->front().index()))
+ m_peerChunks.upload_queue()->pop_front();
+
+ if (!m_peerChunks.upload_queue()->empty()) {
+ if (m_peerChunks.upload_queue()->front().index() == m_data.lastIndex) {
+ m_data.bytesLeft -= m_peerChunks.upload_queue()->front().length();
+
+ if (!m_data.bytesLeft)
+ m_data.lastIndex = InitialSeeding::no_offer;
+ }
+
+ return true;
+ }
+
+ m_download->upload_choke_manager()->set_not_queued(this, &m_upChoke);
+ m_download->upload_choke_manager()->set_queued(this, &m_upChoke);
+
+ return false;
+}
+
// Explicit instatiation of the member functions and vtable.
template class PeerConnection<Download::CONNECTION_LEECH>;
template class PeerConnection<Download::CONNECTION_SEED>;
+template class PeerConnection<Download::CONNECTION_INITIAL_SEED>;
}
Index: libtorrent/src/protocol/peer_connection_leech.h
===================================================================
--- libtorrent/src/protocol/peer_connection_leech.h (revision 1045)
+++ libtorrent/src/protocol/peer_connection_leech.h (working copy)
@@ -43,6 +43,19 @@
namespace torrent {
+// Type-specific data.
+template<Download::ConnectionType type> struct PeerConnectionData;
+
+template<> struct PeerConnectionData<Download::CONNECTION_LEECH> { };
+
+template<> struct PeerConnectionData<Download::CONNECTION_SEED> { };
+
+template<> struct PeerConnectionData<Download::CONNECTION_INITIAL_SEED> {
+ PeerConnectionData() : lastIndex(~uint32_t()) { }
+ uint32_t lastIndex;
+ uint32_t bytesLeft;
+};
+
template<Download::ConnectionType type>
class PeerConnection : public PeerConnectionBase {
public:
@@ -59,7 +72,12 @@
inline bool read_message();
void read_have_chunk(uint32_t index);
+ void offer_chunk();
+ bool should_upload();
+
inline void fill_write_buffer();
+
+ PeerConnectionData<type> m_data;
};
}
Index: libtorrent/src/protocol/handshake_manager.cc
===================================================================
--- libtorrent/src/protocol/handshake_manager.cc (revision 1045)
+++ libtorrent/src/protocol/handshake_manager.cc (working copy)
@@ -194,7 +194,7 @@
// We need to make libtorrent more selective in the clients it
// connects to, and to move this somewhere else.
- (!download->file_list()->is_done() || !handshake->bitfield()->is_all_set()) &&
+ (!download->file_list()->is_done() || !handshake->bitfield()->is_all_set() || download->initial_seeding() != NULL) &&
(pcb = download->connection_list()->insert(handshake->peer_info(),
handshake->get_fd(),
Index: libtorrent/src/protocol/peer_factory.cc
===================================================================
--- libtorrent/src/protocol/peer_factory.cc (revision 1045)
+++ libtorrent/src/protocol/peer_factory.cc (working copy)
@@ -55,4 +55,11 @@
return pc;
}
+PeerConnectionBase*
+createPeerConnectionInitialSeed(bool encrypted) {
+ PeerConnectionBase* pc = new PeerConnection<Download::CONNECTION_INITIAL_SEED>;
+
+ return pc;
}
+
+}
Index: libtorrent/src/protocol/peer_factory.h
===================================================================
--- libtorrent/src/protocol/peer_factory.h (revision 1045)
+++ libtorrent/src/protocol/peer_factory.h (working copy)
@@ -43,6 +43,7 @@
PeerConnectionBase* createPeerConnectionDefault(bool encrypted);
PeerConnectionBase* createPeerConnectionSeed(bool encrypted);
+PeerConnectionBase* createPeerConnectionInitialSeed(bool encrypted);
}
Index: libtorrent/src/protocol/Makefile.am
===================================================================
--- libtorrent/src/protocol/Makefile.am (revision 1045)
+++ libtorrent/src/protocol/Makefile.am (working copy)
@@ -10,6 +10,8 @@
handshake_encryption.h
handshake_manager.cc
handshake_manager.h
+ initial_seed.cc
+ initial_seed.h
peer_chunks.h
peer_connection_base.cc
peer_connection_base.h
Index: libtorrent/src/protocol/handshake.cc
===================================================================
--- libtorrent/src/protocol/handshake.cc (revision 1045)
+++ libtorrent/src/protocol/handshake.cc (working copy)
@@ -526,10 +526,16 @@
// The download is just starting so we're not sending any
// bitfield. Pretend we wrote it already.
- if (m_download->file_list()->bitfield()->is_all_unset())
+ if (m_download->file_list()->bitfield()->is_all_unset() || m_download->initial_seeding() != NULL) {
m_writePos = m_download->file_list()->bitfield()->size_bytes();
- else
+ m_writeBuffer.write_32(0);
+
+ if (m_encryption.info()->is_encrypted())
+ m_encryption.info()->encrypt(m_writeBuffer.end() - 4, 4);
+
+ } else {
prepare_bitfield();
+ }
m_state = READ_MESSAGE;
manager->poll()->insert_write(this);
@@ -621,7 +627,7 @@
// still reading the bitfield/extension and postponed it. If we had no
// bitfield to send, we need to send a keep-alive now.
if (m_writePos == m_download->file_list()->bitfield()->size_bytes())
- prepare_post_handshake(m_download->file_list()->bitfield()->is_all_unset());
+ prepare_post_handshake(m_download->file_list()->bitfield()->is_all_unset() || m_download->initial_seeding() != NULL);
if (m_writeDone)
throw handshake_succeeded();
Index: libtorrent/src/protocol/initial_seed.cc
===================================================================
--- libtorrent/src/protocol/initial_seed.cc (revision 0)
+++ libtorrent/src/protocol/initial_seed.cc (revision 0)
@@ -0,0 +1,289 @@
+// libTorrent - BitTorrent library
+// Copyright (C) 2005-2007, Jari Sundell
+//
+// This program is free software; you can redistribute it and/or modify
+// it under the terms of the GNU General Public License as published by
+// the Free Software Foundation; either version 2 of the License, or
+// (at your option) any later version.
+//
+// This program is distributed in the hope that it will be useful,
+// but WITHOUT ANY WARRANTY; without even the implied warranty of
+// MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE. See the
+// GNU General Public License for more details.
+//
+// You should have received a copy of the GNU General Public License
+// along with this program; if not, write to the Free Software
+// Foundation, Inc., 59 Temple Place, Suite 330, Boston, MA 02111-1307 USA
+//
+// In addition, as a special exception, the copyright holders give
+// permission to link the code of portions of this program with the
+// OpenSSL library under certain conditions as described in each
+// individual source file, and distribute linked combinations
+// including the two.
+//
+// You must obey the GNU General Public License in all respects for
+// all of the code used other than OpenSSL. If you modify file(s)
+// with this exception, you may extend this exception to your version
+// of the file(s), but you are not obligated to do so. If you do not
+// wish to do so, delete this exception statement from your version.
+// If you delete this exception statement from all source files in the
+// program, then also delete it here.
+//
+// Contact: Jari Sundell <jaris@ifi.uio.no>
+//
+// Skomakerveien 33
+// 3185 Skoppum, NORWAY
+
+#include "config.h"
+
+#include "download/choke_manager.h"
+#include "download/chunk_statistics.h"
+
+#include "initial_seed.h"
+#include "peer_connection_leech.h"
+
+namespace torrent {
+
+PeerInfo* const InitialSeeding::chunk_unsent = (PeerInfo*) 0;
+PeerInfo* const InitialSeeding::chunk_unknown = (PeerInfo*) 1;
+PeerInfo* const InitialSeeding::chunk_done = (PeerInfo*) 2;
+
+InitialSeeding::InitialSeeding(DownloadMain* download) :
+ m_nextChunk(0),
+ m_chunksLeft(download->file_list()->size_chunks()),
+ m_download(download),
+ m_peerChunks(new PeerInfo*[m_chunksLeft]) {
+
+ bzero(m_peerChunks, m_chunksLeft * sizeof(m_peerChunks[0]));
+}
+
+InitialSeeding::~InitialSeeding() {
+ unblock_all();
+ delete[] m_peerChunks;
+}
+
+inline bool
+InitialSeeding::valid_peer(PeerInfo* peer) {
+ return peer > chunk_done;
+}
+
+void
+InitialSeeding::clear_peer(PeerInfo* peer) {
+ if (!valid_peer(peer))
+ return;
+
+ peer->unset_flags(PeerInfo::flag_blocked);
+
+ // If peer is still connected, offer new piece right away.
+ if (peer->connection() != NULL)
+ peer->connection()->write_insert_poll_safe();
+}
+
+void
+InitialSeeding::chunk_seen(uint32_t index, PeerConnectionBase* pcb) {
+ // When we have two other seeds, trust that the download will
+ // be sufficiently seeded and switch to normal seeding. This is
+ // mainly for when the user accidentally enables initial seeding.
+ if (m_download->chunk_statistics()->complete() > 1)
+ complete(pcb);
+
+ PeerInfo* peer = pcb->mutable_peer_info();
+ PeerInfo* old = m_peerChunks[index];
+
+ // We didn't send this chunk. Is someone else initial seeding too?
+ // Or maybe we restarted and the peer got this chunk from someone
+ // we did send it to. Either way, we don't know who it belongs to.
+ // Don't mark it done until we see it from someone else, though.
+ if (old == chunk_unsent) {
+ m_peerChunks[index] = chunk_unknown;
+ return;
+ }
+
+ if (old == peer || old == chunk_done)
+ return;
+
+ // We've seen two peers on the swarm receive this chunk.
+ m_peerChunks[index] = chunk_done;
+ if (--m_chunksLeft == 0)
+ complete(pcb);
+
+ // The peer we sent it to originally may now receive another chunk.
+ clear_peer(old);
+}
+
+void
+InitialSeeding::chunk_complete(uint32_t index, PeerConnectionBase* pcb) {
+ clear_peer(m_peerChunks[index]);
+ m_peerChunks[index] = chunk_unknown;
+ chunk_seen(index, pcb);
+}
+
+void
+InitialSeeding::new_peer(PeerConnectionBase* pcb) {
+ PeerInfo* peer = pcb->mutable_peer_info();
+ if (peer->is_blocked())
+ peer->set_flags(PeerInfo::flag_restart);
+
+ // We don't go through the peer's entire bitfield here. This eliminates
+ // cheating by sending a bogus bitfield if it figures out we are initial
+ // seeding, to drop us out of it. We should see HAVE messages for pieces
+ // it has that we were waiting for anyway. We will check individual chunks
+ // as we are about to offer them, to avoid the overhead of checking each
+ // peer's bitfield as well. If it really was cheating, the pieces it isn't
+ // sharing will be sent during the second round of initial seeding.
+
+ // If we're on the second round, don't check
+ // it until we're about to offer a chunk.
+ if (m_peerChunks[m_nextChunk] != chunk_unsent)
+ return;
+
+ // But during primary initial seeding (some chunks not sent at all),
+ // check that nobody already has the next chunk we were going to send.
+ while (m_peerChunks[m_nextChunk] == chunk_unsent && (*m_download->chunk_statistics())[m_nextChunk]) {
+ // Could set to chunk_done if enough peers have it, but if that was the
+ // last one it could cause initial seeding to end and all connections to
+ // be closed, and now is a bad time for that (still being set up). Plus
+ // this gives us the opportunity to wait for HAVE messages and resend
+ // the chunk if it's not being shared.
+ m_peerChunks[m_nextChunk] = chunk_unknown;
+ find_next(false, pcb);
+ }
+}
+
+uint32_t
+InitialSeeding::chunk_offer(PeerConnectionBase* pcb, uint32_t chunkDone) {
+ PeerInfo* peer = pcb->mutable_peer_info();
+
+ // If this peer completely downloaded the chunk we offered and we have too
+ // many unused upload slots, give it another chunk to download for free.
+ if (peer->is_blocked() && chunkDone != no_offer && m_peerChunks[chunkDone] == peer &&
+ m_download->upload_choke_manager()->size_total() * 10 < 9 * m_download->upload_choke_manager()->max_unchoked()) {
+ m_peerChunks[chunkDone] = chunk_unknown;
+ peer->unset_flags(PeerInfo::flag_blocked);
+
+ // Otherwise check if we can offer a chunk normally.
+ } else if (peer->is_blocked()) {
+ if (!peer->is_restart())
+ return no_offer;
+
+ peer->unset_flags(PeerInfo::flag_restart);
+
+ // Re-connection of a peer we already sent a chunk.
+ // Offer the same chunk again.
+ PeerInfo** peerChunksEnd = m_peerChunks + m_download->file_list()->size_chunks();
+ PeerInfo** itr = std::find_if(m_peerChunks, peerChunksEnd,
+ std::bind2nd(std::equal_to<PeerInfo*>(), peer));
+ if (itr != peerChunksEnd)
+ return itr - m_peerChunks;
+
+ // Couldn't find the chunk, we probably sent it to someone
+ // else since the disconnection. So offer a new one.
+ }
+
+ uint32_t index = m_nextChunk;
+ bool secondary = false;
+
+ // If we already sent this chunk to someone else, we're on the second
+ // (or more) round. We might have already found this chunk elsewhere on
+ // the swarm since then and need to find a different one if so.
+ if (m_peerChunks[index] != chunk_unsent) {
+ secondary = true;
+
+ // Accounting for peers whose bitfield we didn't check when connecting.
+ // If the chunk stats say there are enough peers who have it, believe that.
+ if (m_peerChunks[index] != chunk_done && (*m_download->chunk_statistics())[index] > 1)
+ chunk_complete(index, pcb);
+
+ if (m_peerChunks[index] == chunk_done)
+ index = find_next(true, pcb);
+ }
+
+ // When we only have one chunk left and we already offered it
+ // to someone who hasn't shared it yet, offer it to everyone
+ // else. We do not override the peer we sent it to, so they
+ // cannot be unblocked, but when initial seeding completes
+ // everyone is unblocked anyway.
+ if (m_chunksLeft == 1 && valid_peer(m_peerChunks[index])) {
+ peer->set_flags(PeerInfo::flag_blocked);
+ return index;
+ }
+
+ // Make sure we don't accidentally offer a chunk it has
+ // already, or it would never even request it from us.
+ // We'll just offer it to the next peer instead.
+ if (pcb->bitfield()->get(index))
+ return no_offer;
+
+ m_peerChunks[index] = peer;
+ peer->set_flags(PeerInfo::flag_blocked);
+ find_next(secondary, pcb);
+ return index;
+}
+
+bool
+InitialSeeding::should_upload(uint32_t index) {
+ return m_peerChunks[index] != chunk_done;
+}
+
+uint32_t
+InitialSeeding::find_next(bool secondary, PeerConnectionBase* pcb) {
+ if (!secondary) {
+ // Primary seeding: find next chunk not sent yet.
+ while (++m_nextChunk < m_download->file_list()->size_chunks()) {
+ if (m_peerChunks[m_nextChunk] == chunk_unsent) {
+ if (!(*m_download->chunk_statistics())[m_nextChunk])
+ return m_nextChunk;
+
+ // Someone has this one already. We don't know if we sent it or not.
+ m_peerChunks[m_nextChunk] = chunk_unknown;
+ }
+ }
+
+ // Went through all chunks. Continue with secondary seeding.
+ m_nextChunk--;
+ }
+
+ // Secondary seeding: find next chunk that's not done yet.
+ do {
+ if (++m_nextChunk == m_download->file_list()->size_chunks())
+ m_nextChunk = 0;
+
+ if (m_peerChunks[m_nextChunk] != chunk_done && (*m_download->chunk_statistics())[m_nextChunk] > 1)
+ chunk_complete(m_nextChunk, pcb);
+
+ } while (m_peerChunks[m_nextChunk] == chunk_done);
+
+ return m_nextChunk;
+}
+
+void
+InitialSeeding::complete(PeerConnectionBase* pcb) {
+ unblock_all();
+ m_chunksLeft = 0;
+ m_nextChunk = no_offer;
+
+ // We think all chunks should be well seeded now. Check to make sure.
+ for (uint32_t i = 0; i < m_download->file_list()->size_chunks(); i++) {
+ if (m_download->chunk_statistics()->complete() + (*m_download->chunk_statistics())[i] < 2) {
+ // Chunk too rare, send it again before switching to normal seeding.
+ m_chunksLeft++;
+ m_peerChunks[i] = chunk_unsent;
+ if (m_nextChunk == no_offer)
+ m_nextChunk = i;
+ }
+ }
+
+ if (m_chunksLeft)
+ return;
+
+ m_download->initial_seeding_done(pcb);
+}
+
+void
+InitialSeeding::unblock_all() {
+ for (PeerList::const_iterator itr = m_download->peer_list()->begin(); itr != m_download->peer_list()->end(); ++itr)
+ if (itr->second->is_blocked())
+ itr->second->unset_flags(PeerInfo::flag_blocked);
+}
+
+}
Index: libtorrent/src/download/download_main.cc
===================================================================
--- libtorrent/src/download/download_main.cc (revision 1045)
+++ libtorrent/src/download/download_main.cc (working copy)
@@ -42,8 +42,11 @@
#include "data/chunk_list.h"
#include "protocol/extensions.h"
#include "protocol/handshake_manager.h"
+#include "protocol/initial_seed.h"
#include "protocol/peer_connection_base.h"
+#include "protocol/peer_factory.h"
#include "tracker/tracker_manager.h"
+#include "torrent/download.h"
#include "torrent/exceptions.h"
#include "torrent/data/file_list.h"
#include "torrent/peer/connection_list.h"
@@ -56,6 +59,8 @@
#include "chunk_statistics.h"
#include "download_info.h"
#include "download_main.h"
+#include "download_manager.h"
+#include "download_wrapper.h"
namespace torrent {
@@ -67,6 +72,7 @@
m_chunkSelector(new ChunkSelector),
m_chunkStatistics(new ChunkStatistics),
+ m_initialSeeding(NULL),
m_uploadThrottle(NULL),
m_downloadThrottle(NULL) {
@@ -194,10 +200,49 @@
connection_list()->erase_remaining(connection_list()->begin(), ConnectionList::disconnect_available);
+ if (m_initialSeeding != NULL)
+ delete m_initialSeeding;
+
priority_queue_erase(&taskScheduler, &m_taskTrackerRequest);
}
+bool
+DownloadMain::start_initial_seeding() {
+ if (!file_list()->is_done())
+ return false;
+
+ m_initialSeeding = new InitialSeeding(this);
+ return true;
+}
+
void
+DownloadMain::initial_seeding_done(PeerConnectionBase* pcb) {
+ if (m_initialSeeding == NULL)
+ throw internal_error("DownloadMain::initial_seeding_done called when not initial seeding.");
+
+ // Close all connections but the currently active one (pcb).
+ // That one will be closed by throw close_connection() later.
+ if (m_connectionList->size() > 1) {
+ ConnectionList::iterator itr = std::find(m_connectionList->begin(), m_connectionList->end(), pcb);
+ if (itr == m_connectionList->end())
+ throw internal_error("DownloadMain::initial_seeding_done could not find current connection.");
+
+ std::iter_swap(m_connectionList->begin(), itr);
+ m_connectionList->erase_remaining(m_connectionList->begin() + 1, ConnectionList::disconnect_available);
+ }
+
+ // Switch to normal seeding.
+ DownloadManager::iterator itr = manager->download_manager()->find(m_info);
+ (*itr)->set_connection_type(Download::CONNECTION_SEED);
+ m_connectionList->slot_new_connection(&createPeerConnectionSeed);
+ delete m_initialSeeding;
+ m_initialSeeding = NULL;
+
+ // And close the current connection.
+ throw close_connection();
+}
+
+void
DownloadMain::update_endgame() {
if (!m_delegator.get_aggressive() &&
file_list()->completed_chunks() + m_delegator.transfer_list()->size() + 5 >= file_list()->size_chunks())
Index: libtorrent/src/download/download_main.h
===================================================================
--- libtorrent/src/download/download_main.h (revision 1045)
+++ libtorrent/src/download/download_main.h (working copy)
@@ -65,6 +65,7 @@
class TrackerManager;
class DownloadInfo;
class ThrottleList;
+class InitialSeeding;
class DownloadMain {
public:
@@ -95,6 +96,10 @@
have_queue_type* have_queue() { return &m_haveQueue; }
+ InitialSeeding* initial_seeding() { return m_initialSeeding; }
+ bool start_initial_seeding();
+ void initial_seeding_done(PeerConnectionBase* pcb);
+
ConnectionList* connection_list() { return m_connectionList; }
FileList* file_list() { return &m_fileList; }
PeerList* peer_list() { return &m_peerList; }
@@ -157,6 +162,7 @@
Delegator m_delegator;
have_queue_type m_haveQueue;
+ InitialSeeding* m_initialSeeding;
ConnectionList* m_connectionList;
FileList m_fileList;
1 Comment »
One Response to “rTorrent: Erst-Seeding”
on 26 Apr 2009 at 18:50 1.webkatalog said …
Good informations, keep up the good work.