Reindex 54,000 posts

Question

After inserting a large number of Q and A directly into qa_posts, reindexing takes hours... or even days?

1. minute: 750 entries indexed

2. minute: 450 entries indexed

3. minute:  130 entries indexed

....

9th minute 100 entries indexed

.......

( using 90.000 entries in qa_words and 120.000 in qa_tagwords )

 

To much for this script?

Answer 1

The main factor will be the speed of your database - are you using shared hosting?

In any event, you can just leave the reindexing running along in the background - posts are only unindexed shortly before reindexing so search on your site will continue to work fine, even while reindexing is going on.

Comments

No, I am testing on my own engines. Guess I should do proper importing / inserting data including words and tags. Have to take a look at what indexing is actually really doing. Can I use php/cron for indexing in the background?

---

Yes, you can certainly do it in the background with a little bit of coding - see qa-app-recalc.php from this line:

$posts=qa_db_posts_get_for_reindexing($next, 10);

---

gidgreen, are you aware that reindexing can cause problems on the front end of the site? When I have done it several errors appear on the frontend.

---

I wasn't aware of this - what sort of errors?

---

I don't remember exactly but they may have been locking errors when reading/posting. Or session errors when viewing pages. I'll check and let you know. But it also seems to slow the site down a bit while it's indexing.

---

Perhaps these were the deadlock errors that are now resolved in Q2A 1.5? But yes, it will certainly slow down the site - not much can be done about that.