Search function is useless... :-(

[Tristan Perry]

SMF Version: SMF 1.1 RC3
Hey all,
Because my forums are fairly big I was advised to switch to full text searching. However now the search function is pretty crappy - it simply doesn't return as many threads as it should. As outlined by one of my members:

Heya,

Search doesn't seem to dig up threads for me anymore. I'm not sure what I'm doing incorrectly. But this has come up a few times now, where I look for old information, cannot find it with search, but if I browse endlessly, I eventually find it and wonder why Search couldn't tag it.

Example, I search the Necron forum for the words "deep strike," and specially look for posts my say, myself for example. I've written the word deep strike many times in the board; yet it finds zero threads, or often times, just one thread. Another example, if I search the Necron board for "Oblits" via posts by my user name, it comes up with nothing. Yet, I can point to the thread where I wrote it several times. It simply does not show up.

This happens on every board. I've tried to make it more specific. Even searched exactly sentences that I copy & pasted with quotations from a thread, and it still fails to find it.

-- Specifically, try to use search, in various ways, to find this thread and in particular, my post there. I've not been able to have it come up, with multiple search methods at all.

I'm at a loss, as to why?

Any ideas? I've noticed this a lot. Searching for a term that is in many posts won't return many posts, it's quite odd?
Thanks,
Tau Online



Space used by forum messages in the database: 279,884.98 KB
Space used to index messages in the database: 142,680 KB

Search index: Fulltext index
Index: already created [remove fulltext index]
Size: 142,680 KB

Force the use of a search index:    NO
Match whole words only:       NO

[impala68]

try turning on the "force use of a search index"


but looking at your forums it looks like you got it working

[Tanks]

it only search the section you are in when you use the search ? try goin to forum index and search from there.. then try go in to one of the boards and do a search from there.. different results ?

[Oldiesmann]

When I searched for "deep strike" (without the quotes), I got 32 pages of results. The odd thing is though, it said that the relevance of every result was 0% - what do you have your search weights set to?

[Tristan Perry]

try turning on the "force use of a search index"


but looking at your forums it looks like you got it working

Yeah I've enabled that option... not too sure if it's helped at all.

it only search the section you are in when you use the search ? try goin to forum index and search from there.. then try go in to one of the boards and do a search from there.. different results ?

Yeah I get different results. That means it's sort of working correctly then?

When I searched for "deep strike" (without the quotes), I got 32 pages of results. The odd thing is though, it said that the relevance of every result was 0% - what do you have your search weights set to?

Yeah I think that's more than it was before (probably due to enabling "force the use of full text index"?). The relevances are odd...

Relative search weight for number of matching messages within a topic (?):         30%
Relative search weight for age of last matching message (?):       25%
Relative search weight for topic length (?):       20%
Relative search weight for a matching subject (?):       15%
Relative search weight for a first message match (?):       10%
Relative search weight for a sticky topic (?):       0%

I don't think I've ever changed them.



An example I've found is that if I'm trying to return http://forums.tauonline.org/index.php?topic=32651.0 as a result, searching for "Keeps logging me out" or "the site has been logging me out randomly" doesn't return it (even though they are both featured in that thread?)

[Tristan Perry]

Sorry to keep bumping, although full text searching is still not working well, and even when going back to searching with no search index not many results are being found.

EDIT: I just did a search for "trace route" and got a full page's worth of results. I did exactly the same query again ("trace route") and got two results? Currently using a full-text isn't too effective, and not using an index crashes/locks up my forums

[Tristan Perry]

Bumpages

[acdouglas]

SMF Version: SMF 1.1 RC3
Hey all,
Because my forums are fairly big I was advised to switch to full text searching. However now the search function is pretty crappy - it simply doesn't return as many threads as it should.

Any ideas? I've noticed this a lot. Searching for a term that is in many posts won't return many posts, it's quite odd?

I have exactly the same problem with my SMF v1.0.8 installation.  I've tried everything I could think of to straighten the problem out, inlcuding posting the problem to this forum, but nothing.  The Search function simply refuses to behave like a true Search function.

ACD

[Tristan Perry]

Yeah, it's annoying! How large is your forum (how many posts)?

[Tristan Perry]

*Bump*

[Oldiesmann]

Have you tried switching to a custom search index instead? I'm not sure what could be causing your problems.

SMF Version: SMF 1.1 RC3
Hey all,
Because my forums are fairly big I was advised to switch to full text searching. However now the search function is pretty crappy - it simply doesn't return as many threads as it should.

Any ideas? I've noticed this a lot. Searching for a term that is in many posts won't return many posts, it's quite odd?

I have exactly the same problem with my SMF v1.0.8 installation.  I've tried everything I could think of to straighten the problem out, inlcuding posting the problem to this forum, but nothing.  The Search function simply refuses to behave like a true Search function.

ACD

The search functionality in 1.0.x is a lot different than in 1.1 (no search index was used in 1.0.x).

I'll have a developer look at this since I don't know what could be causing the issues.

[Tristan Perry]

Have you tried switching to a custom search index instead? I'm not sure what could be causing your problems.

I'm creating one now. It'll probably take a while, and so I'll post back here with the results when I get a chance to test things out.
Thanks a lot
Tau Online

[acdouglas]

Yeah, it's annoying! How large is your forum (how many posts)?

Tiny.  Only 2859 posts from 42 users since 9 August when the forum was started.

ACD

[acdouglas]

I have exactly the same problem with my SMF v1.0.8 installation.  I've tried everything I could think of to straighten the problem out, inlcuding posting the problem to this forum, but nothing.  The Search function simply refuses to behave like a true Search function.

ACD

The search functionality in 1.0.x is a lot different than in 1.1 (no search index was used in 1.0.x).

I'll have a developer look at this since I don't know what could be causing the issues.

Thank you, Oldiesmann.  That would be greatly appreciated.

(BTW, I caught the responses to this thread only by chance.  I've Notification enabled for the thread, but received no eMail that responses were made.  I just checked my eMail address in my Profile with a test message, and everything worked OK, so that's not the problem.)

ACD

[Tony Reid]

Can you try placing compuarts attached Search.php in your Sources directory please.

I can assure you, search has always been one of our highest priorities. There hasn't been a version that didn't have improvements to the search engine. But at the same time, it's also the most complex function of the forum, especially when striving to have an optimimum speed and high quality results at the same time.

The problem you're describing appears to be a known bug that has been fixed in the CVS. Try the attached version of Search.php.

! Search wasn't always including the subject matches properly. (Search.php)
! Search results could in some cases get a relevance above 100%. (Search.php)

[acdouglas]

The file is noted (in the source) for SMF v1.1.  Will it also work for SMF v1.0.8 (the version of SMF I'm using)?

ACD

[Tony Reid]

No I dont think so.

[Compuart]

Indeed the above file cannot be used for 1.0.8.

I'm not entirely sure btw if it will solve the problems you're describing, but it's a start to try. There are a few issues with fulltext making it a work sloppy on boards with large amounts of posts. For that reason, I've written the custom index, which should work better, though has the unavoidable disadvantage of not being a native index.

[Tristan Perry]

The search function doesn't operate correctly with using that attached file, either I get the same search results/behaviour as with the other script.

Compuart: if you need an admin account or FTP access to my forum just drop me a line

[Tristan Perry]

I've been running the create a custom index tool for a few hours for the past few days (probably over 12 hours worth), and it's still on only 47.72%... plus my forum's database has doubled in size and so I can't create backups of it (as easily) anymore. Is there any solution to this other than the custom index and I feel it'll be too much to create/maintain?
Thanks,
Tau Online