Why over 2000 smileys is necessary for some people and should be taken seriously

[tpgames]

 (Anger directed more at people who call us "bozos" if we use more than a token amount of smileys that is low and at programmers who think that we do not need more than 1500 smileys because more than that is just stupid or a headache). Keep in mind, I spent years trying to find a forum that would actually allow my smileys my way on a forum. I have special needs that require me to use smileys outside of the box in order to process the information on the page faster. A sea of text is just that - a sea or ocean of text that makes little sense to me. Also, Kanji and Kana is just EASIER for Non-Japanese with Non-Japanese keyboards and computers, to just use smileys. I can't post Kana without stupid windows trying to turn it into Kanji. I might be doing something wrong here, but I am a noob to writing in Kana and Kanji on a keyboard.

If SMF goes the way of those other arrogant forums (that I have tried), then I will have to ditch the use of forums and go with something else less desirable (something we have already tried and didn't like - hard coding html pages, Mind Maps, TreeSheets, Real White Boards!) I prefer a private forum on my website where by only my people can see it.  The issues were: Must be public, must be English only, rated G or PG (no MA games which my friend does, but not me), no stories, or no games depending on the slant of the forum. NO child of child of child of child boards (except Yabb). Smileys limited to only what we give you (for free online, not open software YaBB; but phpBB I had major issues with getting my smileys on there, but this was a few years back.)


1. If I use my forum to plan a game I do not want to upload an image every single time! I can use the smiley character that is not a "Eeks!!! - I see something quick hide it 16x16 size!" if I have freedom of number of smileys i use. We use the forum to plan oodles of games, stories, VNs, CYOA, etc. None that really becomes public, but might some day.

My smileys range in size from 8x16 all the way up to 200x200 but most of them do not go over 100px in size. Only a few are 780px long, but only 6px tall and used to separate content in a visual way such that I can tell by the colour of that line, what content is going to follow.

We use these smileys for bullets sometimes because I hate dots and find them them hard to see. Plus, if everything is dots, then I can't say that these bullets are dealing with the quest part of the game, and these bullets deal with the fighting aspect, or the puzzler part of game.

If I have to go to pb to get image code to put in my forum with every single post I will go through the roof! I even seriously considered downgrading SMF to a older version that just worked, but I need it to work with the bleeding edge PHP. I used to use pb for my smileys and quit because that took so long, plus their ads were sometime viral.

2. Planning stories. Yes, some of us are actually writers, as opposed to just using forums for chit chat, programming and other dialogue that does not require smileys at all.

What is attached is a stupid example of what I do on my forum. I say stupid, because it was a quickly done example, instead of an actual example. So yes, I am calling my example stupid.

And yes, the 65kb issue that limits smileys to 1500 can be fixed. There has to be a way. Some forums all they do is smileys and they are all about smileys and how cool they are. And they want you to upload your smileys to show them off and share. What forum software do they use? Even YaBB I think had upper limits. I think it was the same 1500, but don't quote me on that, as its been awhile since I used YaBB. I switched back to SMF.

Please See Attachments as an example as to what I do. BTW, I have some boards 6 or 7 levels deep, but when I plan out my CYOA, that board could be up to 25 layers deep. At 25 layers, I'd use a device such as "board layers 26 - 50". The actual title will be something that just tells me it comes after whatever was at layer 25. And, will in reality start back at layer 2 or 3.
Thanks for bearing with me and reading all of this! My friend and I greatly appreciate it!

[Arantor]

What limit, exactly, do you think you are hitting at 64K?

[nend]

Processing and memory come to mind.

Me myself have ditched the SMF smiley system for a more robust solution IMHO. I started using Unicode emoticon codes and convert them to their respectful image. This is more CPU and memory friendly, instead of adding a massive array of smileys that would have to be stored and processed.

I can see how this setup doesn't look applicable in your case, just wanted to point out the obvious when it comes to smiley support.

[Arantor]

Ohhhhhhhhhhhhh I just remembered what the issue is and why it's 64K. Especially since I figured out why it's 64K and why I said it shouldn't be changed.

The longer that limit goes, the slower it gets, and it will get exponentially slower the more smileys you have. If the smiley parsing is switched out to be case sensitive rather than case insensitive, it's solvable by converting the single massive regex out for a strtr (so you don't replace changes you already changed, something you could do with str_replace because that won't execute in parallel while the regex will, and the strtr checks itself from doing that)

EDIT: even if this is changed it must not be done for everyone because this will detriment the majority at the expense of the few. 2k smileys is not a common case, this is the third case ever of more than 1000 I've seen and only the second to trip the 1500 danger point.

[Kindred]

exactly this...

tpgames...   please realize that what you suggest is a definite edge case scenario that 99.9999999999999999% of the sites would never, ever, ever encounter.
As Arantor says - in over a decade, we have seen 3 cases of people overusing smilies.

So, what you think you need for your site is one thing...   and might be doable by someone for you as a custom build --- but there are actually reasons why the limit is currently in place and it is unlikely that it would be changed in the base software for such an edge case... especially since such a change might have deleterious effects on everyday forums who are nowhere near the smilies limits.

[tpgames]

Thanks for all the info! But.... and don't take offense at this as I just need to vent a little.  I get the programming issue and the kb limit. I think it is stupid in this era of 20TB hard drives for "people at home who do not even need 20 TB hard drives" we can't solve the kb limit because speed is an issue. We be programmatically behind the hard drive space accommodation given how media intensive our culture has become with all those video ads, videos, animated gifs and so on. (I mean,  programming knowledge has not arrived where it needs to in order to accommodate our new era of resource intensive things.) Imagine if it was the norm for videos instead of smileys? This does NOT mean that you are lacking skill. No, I just think we've not come up with the programming yet and that forum software is hugely behind what youtube even has.

And these answers is exactly why I detest being the minority (again no offense to you guys!) I have always been told that I should never...., or could never..., or should be like the majority. I have spent 51 years of pure hell being told "NO!" because you are a bleeding minority and we only care about the privileged majority who get whatever they want! We do not cater to anyone who is not apart of the majority - and includes food allergies, art projects in school, what classes I can take due to lottery hates me, what special topic I can have, whether I have friends or not due to stuttering brain disorder and so on). I get it. This is not just SMF - this has been every single year of my life in 90% of my life. I am trying to just say this so that I can then move on and roll with the punches and just assume that 100% of what I want is a NO and that I should never want anything because I won't get it anyway! (Including kids, marriage, a real job, a brain that works, ability to write and publish books, learn math, program for real, and so on.) I can't even afford to hire a programmar and unfortunately my friend who knows OLD school programming refuses to help me because he now hates programming. He is 100% self-centered to a fault anyways. 

This is also why I hope I can con my friend who hates programming with a passion (burned out) to learn PHP so that he can fix this issue for me. I only have 2 people on my forum so I am not that concerned with speed. I'd even take a mod that solved this issue. I won't be having 10,000 smileys. I'd limit it to no more than 3000 max. I might consider having 2 stupid forums on my site just because 100% of ALL FORUMS (not just SMF) think the minority should just die to there wants and needs. I wish I did not have Fetal Alcohol Syndrome because if I didn't, I would have programmed my own forum ages ago and wouldn't even be on any one elses forum. I wouldn't have a need. And, because my forum is so private such that crackers can't really find it that easily, let alone join, I wouldn't have to worry so much about security, but still would have to consider php version of server hosting it.

With this, I guess case closed, I will never ask for anything again for a feature because no one wants it anyway. on me!

[Kindred]

tpgames....

that is not at all what we said.

We have to build a software that runs properly for users.
We implement features which will be used by or are of interest to the majority of the people who use the forum.
We remove features which we find are underutilized
We don't add features which won't be used by the majority.

HOWEVER....  we have this wonderful tool called the package manager which allows authors to build mods which can do something as small as change one icon or alter the very structure of the software....   This allows admins/users who WANT a specific feature that doesn't make it into the core product to ADD that feature...

We never said that you could not do what you are asking...
we just indicated that it was unlikely to be done within the core product (and do note, you posted this request in the FEATURE REQUESTS board -- which is the area for requests of features to be added to the core product.)
This sort of request is definitely the sort of thing that should be done as a mod -- and perhaps, if you post in the mod requests board, you might find someone who would be willing to code it for you...

[Arantor]

It isn't about space. 64K is a tiny amount.

The problem is CPU time. The smileys get put into what's called a regular expression. And it goes through your post, one letter at a time, trying to match it against any of the possible smileys.

I'm guessing most of your smileys start with a : but not all of them e.g. so for every single letter in your post, it has to compare against the first character in every smiley. If none of them match, that's still 2000 checks for every letter of your post. 1000 letters in a post (barely 200 words) creates a best possible case of 2,000,000 checks when you have NO smileys.

If you happen to have a : in your post either because you have a smiley, or just typing it, it's still going to go through that 2000 smiley list. Only if it matches on one, it then looks at the second character to see if it matches. If not, it moves onto the next possible smiley. Suddenly you're not talking 2,000,000 matches. You're now potentially exponentially increasing it, into the thousands of millions. Consider, for example, if you have a bunch of smileys that are all :a smileys. If it gets through the : choices and then to : followed by a, it has to check every possible choice, and not stop until it's matched or found no possible matches.

This is why it's a problem. 64K of regular expression is typically server flattening levels of regular expression. It's why PHP even has internal protections against abusing it, but the way this is written, it sidesteps PHP's own protections on backtracking matches.

Oh, and remember this is per post, typically 20 posts a page.

And before you went on your venomous rant about it, I was prepared to actually write you a mod that would do it if having case sensitive was OK, but I guess I should just crawl into a hole because I know nothing and it should be simple and it's just an arbitrary limit. I wasn't even saying no, I was saying "yes on condition of X" but this isn't good enough.

Now I remember why I was glad life had kept me busy enough not to come here.

[tpgames]

I had a brain storm after a very long walk to cool off.
How stupid of an idea is it, to make it so that members could switch smiley sets while writing a post? I could then have no more than 1500 smileys per "smiley set". Only the Kanji folders would get that many smileys. (UPDATE: Does not work because smiley sets can't have more than one smiley set link per post, currently. Same with below. )
OR
After writing a post with smiley set 1, be able to go back and modify and choose smiley set 2? So far I am not getting this idea to work as I am getting errors "smiley with this code already exist" but smiley doesn't exist in set unless I add it even though I didn't change anything. (Was planning on using existing code and just swap out the smileys behind the scenes, but keep all name and file types the same.)

UPDATE: I did try to switch smiley sets (with 3 sets having the same image names) but 1. I had to use different codes 2. images then would not be found when I switched sets again. The previous image that showed up on post, would then not show up. Thus, program, must assume that each post has 1 smiley set link. And how come I have this sneaking suspicion, that I've thought of this whole idea before but was told it wouldn't work (with good reasons that I do not remember?) It must have been before YaBB days. Oh well, I thought if I could just switch smiley sets, then no one would have to bother with me, and no mod would be needed and I'd just have to do all the work. Oh well, I tried.

Would this still have the same 64kb issue or is that 64kb issue only PER smiley set?
Thanks! (And after this I will shut up about it.)

BTW, some smileys start with V, others Z, b or some other character that is added to the name of the smiley. thus, I could do 26 different letters to start the smileys with (and would gladly change some of my z+ name smileys to be some other letter).

IF with no smileys, it is checking against every single character then wouldn't we want to solve this regardless? (Unless normal HTML pages or blog pages does this too! I never learned this. If they do, then just skip the rest of this paragraph. lol) I mean, ignore smileys! What I am understanding (perhaps wrongly) that if we were to post chapters in each post, that it would be going through thousands of checks. Again, if static html pages and blog pages do this too, then forget about it.

And my venting wasn't really about just SMF. My entire life has just been against me in so many areas that I just do not know what to do anymore. I hate it. I wish I could get something my way! I get 99% NOT my way. SMF still exists. That is 1 thing.

Keep in mind, I have a mucked up brain! I am stupid in the head - literally! Its because my stupid grandfather murdered my grandmother and then committed suicide in front of my 9 yr old mother. She grew up to have abandonment so severely, that she had ONE drink on her honeymoon because she thought she would be abandoned if she didn't, and it turned out she was 9 days pregnant with me. The result was a mucked up brain with lots of damage that prevent me from learning programming well enough to do anything with it. It took me 10 years to finally be able to do simple HTML. I still haven't been able to figure out how to do original JavaScript. Its over my head. I am using w3school's php (which I just started). I am hoping my brain can figure out php to a point. My brain only randomly shows up. If I could get it to show up for a full hour at a time, I could very slowly learn programming. It only shows up for ten minutes max at a time, then kicks out what I figured out.

Yes, I am familiar with CPU time. I get that server load would hate me. I do have to keep that in mind. We only have two people, so we aren't concerned about load times for us. This is THAT private of a board. lol

@Arantor
I would gladly accept a mod. Case sensitive does NOT bother me. I'll take anything that works. Yes, I am that frustrated.  (And yes if you want me to post this request in the mod request board, I will.)

[tpgames]

Me doh! Obviously, all the smileys are shared across the sets. I knew this!  Thus, its is a doh that smiley Kanji 1 from Kanji set will not show up on forum when user is on  fantasy smiley set. And, switching smiley sets to view a post will not work because I would be mixing sets in the same post. lol Thus, I am back to the mod: to allow more than 1500 smileys.