Jump to content

Welcome to DevFuse Forums

Sign In  Log in with Facebook

Create Account
Welcome to DevFuse Forums, like most online communities you must register to view or post in our community, but don't worry this is a simple free process that requires minimal information for you to signup. Be apart of DevFuse Forums by signing in or creating an account.
  • Start new topics and reply to others
  • Subscribe to topics and forums to get email updates
  • Get your own profile page and make new friends
  • Send personal messages to other members.
Guest Message by DevFuse

(View All Products)Featured Products

  • Donations

    Help fund your forum with donations, setup goals and track member donations. Offer rewards for members donating.
  • Timeslips

    Have your members submit their race times and share with others.
  • Videos

    Allows your members to submit their own videos for community viewing. Support is included for all the major video sites.
  • Forms

    Build your own forms for your members without coding experience. Support included for pm, email and topics.
  • Collections

    Build a community database of items for your members. Full features custom fields included.


[IPS] Unofficial Multi-Threaded UTF-8 Converter

This topic has been archived. This means that you cannot reply to this topic.
No replies to this topic

#1 News Bot

News Bot

    Dedicated Member

  • Members
  • PipPipPipPipPip
  • 1,909 posts
  • IP.Board Version:N/A

Posted 20 October 2013 - 02:35 PM

With the release of IPB 4 coming in the future, one of the things that is going to hit people with large communities hard is the conversion to UTF-8 if you are not already there.  This can be a huge task, and with bfarber's converter, this works fine, but can be very slow as PHP can only run so quickly as it's bound by a single CPU thread.  On my forum, which is a total DB size of ~10GB, bfarber's single threaded script takes over 48 hours straight to run.  I found this amount of downtime for my live forum to be unacceptable, so I created a fork of his script and used PHP's fork functions to multi-thread it.  With 8 cores assigned to the script, I was able to convert this database in just 5 hours under testing.  When I did my live conversion, I assigned 12 cores and was able to do it in 3 hours.
This script is aimed at people with large databases, where a conversion would normally take a significant amount of time, and people who are power users and understand server maintenance and configuration.
Please ensure you carefully read the included readme.txt file, and the documentation in the PHP file for the settings.
The database I converted was an IPB 3.4.5 database, but since this is based on bfarber's script from 2010, there's no reason this shouldn't work on pretty much any IPB version in the 3.x.x series, but I will only be officially supporting it on 3.4.5.

View Resource