Bug with more 15000 locations

ClassiBase classifieds script Forums General help Bug with more 15000 locations

Tagged: ,

This topic contains 7 replies, has 2 voices, and was last updated by  ClassiBase 2 years, 8 months ago.

Viewing 8 posts - 1 through 8 (of 8 total)
  • Author
    Posts
  • #1638 Reply

    philippe philippe
    Participant

    Hello,

    I have create one new site, but after import 1500 location, i don’t can create new location or category, i have message: Error for save category or Error for save locations.

    Do you have a solution?

    #1644 Reply

    ClassiBase
    Keymaster

    Can you please clarify is it 15000 locations or 1500 ?

    How you add locations via add location form or import locations tool?

    If using import locations tool maybe you can split your locations to 1000 location files and import one file at a time?

    Also you can enable debug mode via “Tools” -> “Maintenance” -> “Debug mode” and see if you are out of memory or execution time issues.

    Please let us know how thins go.

    #1647 Reply

    philippe philippe
    Participant

    Is not a problem in import tool, because when i have more 2000 locations, in Manage -> Locations, i view a blank page, and in database i see all locations.

    What the problem???

    #1648 Reply

    philippe philippe
    Participant

    ????

    #1649 Reply

    ClassiBase
    Keymaster

    How many locations in each level? 1000 in first and 1000 in each or 10 in first level and 1000 in each?

    In any case we will test it on our server and let you know of findings. meanwhile you can enable debug mode and see which queries take long to process.

    Regards.

    #1651 Reply

    philippe philippe
    Participant

    There are approximately 100 and 400 in the lower levels. You want the files?

    #1653 Reply

    ClassiBase
    Keymaster

    Ok, We have done some test. We used our local computer with 3GB RAM, win8 PC with Apache 2, PHP 5.5. Used Location file with 100 main locations, 400 sublocations in each main location, totaling 40K locations.

    Here are findings:
    When using bulk import for locations script processes until it times out. in our case it is 30 seconds. In first run it imports 1500 locations. on consecutive imports of same file it continues importing non existing locations but import quantity is reduces up to 1000, then 500, 300 etc. That is why it is better to split your locations to 1000 location files and import in chunks. Also if not all 1000 locations imported (you will know in case script opens blank page, file is not fully imported), you will have to re-import same file until you redirected to locations page after import.

    We tested our system with 10K (it is 10 000) locations, we did’t import all 40K locations because end result can be predicted by viewing 10K locations. Script uses cache for locations and categories for not requesting them from DB each time.

    Reading 10K locations with names from DB and storing to cache takes on average 1.1 second and uses 35MB system memory. On consecutive page load reading locations takes 0.17 seconds and same amount of memory 35MB.

    This means if you want to use more than 2K locations it will be better using dedicated server with 2 or 4 GB memory. Because 100 simultaneous users with 35MB memory use will be 3.5 GB memory needed by PHP only.

    According to this research we can suggest several solutions for you:

    1. Use dedicated server with 4GB ram if you want to host site with 10K locations, use 16GB memory server to host site with 40K locations and so on.
    2. Split your locations to countries if possible and create separate website for each country.

    In any case if you want to host a site with 150K cities ( there are around 190 countries and 300K cities/towns in the world) then you definitely need dedicated developer or team of developers/engineers to architect and load balance your servers (should be at least 3 servers), because imagine number of ads hosted on that server and income generated from it.

    #1656 Reply

    ClassiBase
    Keymaster

    As you asked for 15K locations, we tested with 15K. It uses 54MB memory and loads from cache in 0.5 seconds. When adding new location using form all 15K locations are loaded to select box for defining parent location of new location. That may take a while to load as HTML file size is 4MB.

    So for website with 15K it is suggested dedicated server with at least 4GB RAM.

Viewing 8 posts - 1 through 8 (of 8 total)
Reply To: Bug with more 15000 locations
Your information: