[pdf-devel] Re: Fix probable problem with array subscript in ASCII85 decoder

[Scott Fohey]

> Details - We are getting the following warnings while compiling the library.  
> It may indicate a possible
> buffer overflow. The declaration of 'pdf_stm_f_a85_write_outbuff' shall be 
> deleted if it won't be used at all.
>
> base/pdf-stm-f-a85.c:50: warning: 'pdf_stm_f_a85_write_outbuff' declared 
> 'static' but never defined
> base/pdf-stm-f-a85.c: In function 'pdf_stm_f_a85dec_apply':
> base/pdf-stm-f-a85.c:775: warning: array subscript is above array bounds
>

Attached is my fix for the A85 filter.

There was indeed an overrun of an array by one byte. (My compiler
didn't catch this one - I guess I need to update)

The function prototype name didn't match and was corrected.

Flyspray task assigned and closed.

Here are the two changes:

 static pdf_status_t
-pdf_stm_f_a85_write_outbuff (pdf_char_t c, pdf_buffer_t out,
-                             pdf_stm_f_a85_t filter_state);
+pdf_stm_f_a85_write_out (pdf_char_t c, pdf_buffer_t out,
+                         pdf_stm_f_a85_t filter_state);


-#define A85_SPARE_BYTES_LEN 4
+/* If something goes wrong in writing out data, 5 bytes could be leftover */
+#define A85_SPARE_BYTES_LEN 5

--Scott

fix_a85.merge
Description: Binary data